Bug 140242

Can you find the condition that causes the problem? Is there a particular message that causes kmailcvt to freeze, or does it occur for mailboxes larger than a certain size, or ...?

Same problem for me, importing from Evolution 2.8.3 to Kmail 1.9.6 (KDE 3.5.6-0.3.fc6), kmailcvt Qt: 3.3.7, KDE: 3.5.6-0.3.fc6 Fedora-Core, KMailCVT: 3

It hangs just after the first folder has been imported and the windown must be killed.

Mailboxes where originally imported to evolution from ms outlook (win98).
The structure is some nested and contains some huge attachments
(du of .evolution/mail/local is about 402MB) 

I also tried starting at a branch... it imports a few more but with no header information and then hangs the same way.

I think I have found the solution:

The Problem is the file *.ev-summary-meta in the ./evolution/mail/local/ Directory (or any Subdirectory). 

$ tree -L 1
.
|-- Inbox
|-- Inbox.cmeta
|-- Inbox.ev-summary
|-- Inbox.ev-summary-meta
|-- Inbox.ibex.index
|-- Inbox.ibex.index.data


In all Versions of the File filter_evolution_v2.cxx (tags and trunk Directory) are two If terms which filter the Files:

File: filter_evolution_v2.cxx:
...
void FilterEvolution_v2::import(FilterInfo *info)
{
...
  if (temp_mailfile.endsWith(".cmeta") || temp_mailfile.endsWith(".ev-summary") ||
      temp_mailfile.endsWith(".ibex.index") || temp_mailfile.endsWith(".ibex.index.data")) {}
  else {
      info->addLog( i18n("Start import file %1...").arg( temp_mailfile ) );
                importMBox(info, mailDir + temp_mailfile , temp_mailfile, QString::null);
  }
...
}
...

void FilterEvolution_v2::importDirContents(FilterInfo *info, const QString& dirName, const QString& KMailRootDir, const QString& KMailSubDir)
{
...
  if (temp_mailfile.endsWith(".cmeta") || temp_mailfile.endsWith(".ev-summary") ||
      temp_mailfile.endsWith(".ibex.index") || temp_mailfile.endsWith(".ibex.index.data")) {}
  else {
      info->addLog( i18n("Start import file %1...").arg( temp_mailfile ) );
      importMBox(info, (dirName + "/" + temp_mailfile) , KMailRootDir, KMailSubDir);
  }
...
}
...

The Problem is it excludes only the *.cmeta, *.ev-summary, *.ibex.index and *.ibex.index.data but not *.ev-summary-meta files.
The outcome of this is an infinite loop here:

File: filter_evolution_v2.cxx: (tag 3.5.7-version):
...
void FilterEvolution_v2::importMBox(FilterInfo *info, const QString& mboxName, const QString& rootDir, const QString& targetDir)
{
...
  while ( ! mbox.atEnd() &&  (l = mbox.readLine(input.data(),MAX_LINE)) && ((seperate = input.data()).left(5) != "From ")) {
      tmp.file()->writeBlock( input, l );
  }
...
}

The easiest way is to exclude the *.ev-summary-meta too:

if (temp_mailfile.endsWith(".cmeta") || temp_mailfile.endsWith(".ev-summary") ||
    temp_mailfile.endsWith(".ibex.index") || temp_mailfile.endsWith(".ibex.index.data")
    || temp_mailfile.endsWith(".ev-summary-meta")) {}

I hope this is helpful.

Can anybody here please attach a minimal testcase (sample mail which causes kmailcvt to hang)?

I take a look at it and test it in the next days. Thanks.

Created attachment 21548 [details]
minimal testcase

Here is a minimal testcase.
I use for this test Evolution 2.10.1 from Ubunut-live-CD 7.04.
Important is the file Drafts.ev-summary-meta.

I only made a new mail and saved it as Draft. Then I clickt in Inbox. Evolution
saves then the file *.ev-summary-meta.

I have a similar problem when importing from thunderbird. I begin importing my folders, but at one point it kind of freeze. I have many subfolders (Drafts, Factures, Inbox, Sent, Templates) and it always freeze before reaching the "Sent" folder. So in kmail the "Drafts", "Factures" and "Inbox" are imported. I do not know if all the messages were imported though.

I straced kmailcvt. It seems to enter a kind of loop somewhere :
read(9, "\0\0\0\6kmail\0\0\0\0\20anonymous-12234\0\0\0"..., 46) = 46
unlink("/tmp/kde-nicolas/kmailcvtdE2Qic.tmp") = 0
read(3, 0x6402e4, 4096)                 = -1 EAGAIN (Resource temporarily unavailable)
select(13, [3 4 5 7 9 11 12], [], [], {0, 0}) = 0 (Timeout)
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
stat("/tmp/kde-nicolas/", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
open("/tmp/kde-nicolas/kmailcvtNChdHb.tmp", O_RDWR|O_CREAT|O_EXCL, 0600) = 14
umask(0)                                = 022
umask(022)                              = 0
fchmod(14, 0600)                        = 0
getgid()                                = 1000
getuid()                                = 1000
fchown(14, 1000, 1000)                  = 0
fcntl(14, F_SETFD, FD_CLOEXEC)          = 0
fcntl(14, F_GETFL)                      = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(14, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2af9e3892000
lseek(14, 0, SEEK_CUR)                  = 0
fstat(14, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
read(14, "", 4096)                      = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
[...many exactly the same...]
read(13, "tie 1.ZIP\"\r\n\r\nUEsDBBQAAAAIAOCSbC"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "From - Fri Apr 09 14:01:30 2004\r"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
read(13, "+gbXvyrYKu7j2FdNqce/qbZPCA7Fc+gU"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "uLJ0DqtwmAqfTQlEJFpEgwKJePxyolky"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
read(13, "H2Vn+vrpJtr17QfunGpUtLJ85/H81f+7"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "+GYpvW6ABxxW8opotY6phDJUsdO9aGCW"..., 4096) = 4096
[...]
After that it is always this patern repeated:
read(13, "+gbXvyrYKu7j2FdNqce/qbZPCA7Fc+gU"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "uLJ0DqtwmAqfTQlEJFpEgwKJePxyolky"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
with different values in for the read and write, always looks like garbage...

I stoped the strace when the log was about 200MB...

Could it be because the folder in thunderbird was compressed? Or maybe encrypted?

Nicolas, this bug is about evolution. Your problem may perfectly be totally different. Please check: http://bugs.kde.org/show_bug.cgi?id=117335 and http://bugs.kde.org/show_bug.cgi?id=128699 and if you bugs doesn't match with any of them open a new bug report.

Thanks.

SVN commit 746733 by dkukawka:

- fixed variable names
- changed importDirContents() to reuse existing code to import
  mails from the root directory
- ignore *.ev-summary-meta files while import (bug #140242) 

CCBUG: 140242



 M  +24 -27    filter_evolution_v2.cxx  
 M  +1 -1      filter_evolution_v2.hxx  


WebSVN link: http://websvn.kde.org/?view=rev&revision=746733

*** Bug has been marked as fixed ***.