Bug 285676

It's actually akonadi 1.6.2.

I may have the same kind of problem, reported in the bug 285809 : after upgrading to KDE 4.7.3, everytime kmail start synchronizing some specific folders, it hangs handlessly. I indeed saw in the console identical messages to the above about missing files, though. I didn't have this issue with KDE 4.7.2. I've been using akonadi 1.6.2 in both versions of KDE.
Lote that going back to KDE 4.7.2 now doesn't seem to fix the issue.

Another bit of debug info. The network connection was not broken while these messages appeared and kmail had already downloaded and filtered messages before failing again.

And again the retrieval of one message blocked everything else. Of what use is a non-blocking mysql if akonadi itself can only handle one thing at a time?

request for item 40428 "329499" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 40429  there are  1  queues and  0  items in mine 
request for item 40429 still pending - waiting 
processing retrieval request for item 40429  parts: ("RFC822")  of resource: "akonadi_imap_resource_2" 
continuing 
request for item 40429 "329500" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 40430  there are  1  queues and  0  items in mine 
processing retrieval request for item 40430  parts: ("RFC822")  of resource: "akonadi_imap_resource_2" 
request for item 40430 still pending - waiting 
329479 +FLAGS (\Deleted)
329480 +FLAGS (\Deleted)
329482 +FLAGS (\Deleted)
329483 +FLAGS (\Deleted)
329643 FLAGS ($IGNORED)
continuing 
request for item 40430 "329501" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 40431  there are  1  queues and  0  items in mine 
processing retrieval request for item 40431  parts: ("RFC822")  of resource: "akonadi_imap_resource_2" 
request for item 40431 still pending - waiting 
continuing 
request for item 40431 succeeded 
Could not contact query service. 
QStringList Akonadi::NepomukSearch::search(const QString&) Calling blockingQuery() failed! 
posting retrieval request for item 40432  there are  1  queues and  0  items in mine 
processing retrieval request for item 40432  parts: ("RFC822")  of resource: "akonadi_imap_resource_2" 
request for item 40432 still pending - waiting 
continuing 
request for item 40432 succeeded 
posting retrieval request for item 40433  there are  1  queues and  0  items in mine 
processing retrieval request for item 40433  parts: ("RFC822")  of resource: "akonadi_imap_resource_2" 
request for item 40433 still pending - waiting 
continuing 
request for item

Same counts for me!

Akonadi states:
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. 

Kontact/Kmail is nearly unusable cause of endless delays now ;(

I think this regression was introduced only some days/a week ago, since 4.7.2 + akonadi 1.6.2 did not suffer from this. Only after the packages I use got their last branch update (maybe a week before 4.7.3) it started.

The problem starts after I get the following in the edbug output. I can reproduce it by clicking on a folder within a mixed_maildir_resource which itself contains files and sub-folders.

Could it be that akonadi is looking for filenames which do not exist because the emails are stored in an mbox file?

"Der angegebene Ordnername ist leer" translates to "The given folder name is empty".

akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2505966" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13521    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13521") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2532086" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13523    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13523") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2558664" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13525    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13525") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2564837" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13527    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13527") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2593561" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13529    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13529") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2598645" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13531    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13531") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2618353" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer" 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir): "Der angegebene Ordnername ist leer" collection: Collection ID: -13533    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-13533") 
   parent: -33 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 
akonadi_mixedmaildir_resource_0(17634)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "2631595" in collection -33 , "" did not return the expected item. error= 102 , "Der angegebene Ordnername ist leer"

Same for me.

KMail is unusable: my the outbox folder is stuck because of that, so I have to use another mail client to be able to send emails.

I managed to recover my outbox folder.

What probably happened is that the Akonadi database and the maildir directory got desynchronized after a crash of the system (I have troubles with the suspend).

In this situation the akonadi_mixedmaildir_resource seems to get stuck trying to synchronize.

To recover the outbox I had to remove the files that were in the 'new' directory and remove the pim items from the database by hand.

If I manage to reproduce the problem in a simple case I'll post it here.

Cheers
Marco

Ok, I managed to reproduce the problem.

From a fresh account, start Kontact (KMail2 should be enough).
Prepare a directory with this kind of substructure:

testing
└── fldr1
    ├── cur
    ├── new
    └── tmp

From the KMail settings create a "KMail Maildir" account making it pointing to the directory "testing".

Create a new mail and save it as draft.

Copy the file in the "new" subdirectory of the drafts directory (somewhere in ~/.local/share/.local-mail.directory/) to the directory testing/fldr1/new.

From the KMail interface select the folder "fldr1" and synchronize it (F5 or right-click and synchronize). 

Nothing seems to happen, but go to the KMail settings, then accounts, and you will see that the testing account got stuck in the synchronization and is unusable.


The "Maildir" type of account (not "KMail Maildir") doesn't seem to have the problem. If you try the same procedure using a "Maildir" account, it behaves correctly.

Cheers
Marco

Wow! Thanks for finding out the reason and how to reproduce. Great job!

Great job, Marco. This seems to fit exactly my situation.

Marco, does it happen if you use Maildir and not KMail Maildir?

Sorry, I was too fast answering, didn't read it in deep. So it is a mixedmaildir bug.

*** This bug has been confirmed by popular vote. ***

i had the same problem in gentoo after upgrading kde-4.7.2 to 4.7.3.
i've found that downgrading kde-base/kdepimlibs (and all packages, which hove to be downgraded with it) fixes the problem. note, that i've downgrade kdepim-runtime, kdepim-common-libs and kmail earlier and that didn't help

oh, and BTW:
i still have these strange messages like:


akonadi_mixedmaildir_resource_0(15807)/akonadiresource (maildir) RetrieveItemsJob::Private::fetchChangedResult: Store fetch for changed item "1234738150.28656.6pJJr:2,S" in collection -27 , "" did not return the expected item. error= 102 , "Given folder name is empty" 
akonadi_mixedmaildir_resource_0(15807)/akonadiresource (maildir): "Given folder name is empty" collection: Collection ID: -1210    remote ID: "" 
   name: "" 
   url: KUrl("akonadi:?collection=-1210") 
   parent: -27 "" 
   resource: "" 
   rights: QFlags(0x1|0x2|0x4|0x8|0x10|0x20) 
   contents mime type: () 
    CachePolicy:  
   inherit: true 
   interval: -1 
   timeout: -1 
   sync on demand: false 
   local parts: () 
    CollectionStatistics: 
   count: -1 
   unread count: -1 
   size: -1 

i suppose i always had them. but akonadi and kmail are fully functional now, so i'm not sure if that hanging kmail during folder synchronization is related to them

see if is that solution:
http://lists.kde.org/?l=kdepim-users&m=132140480409324&w=2

i don't quite undestand that solution. could you point me to some documentation, which'd answer to these questions:
1. what's the difference between 'kmail folders' and 'local folders' resources?
2. which one should i have after migrating from kmail1?

the suggestion of comment #17 works, but it is a workaround and not a fix. BTW, that's what I did to be able to work.

To answer to comment #18:
1. my understanding of the difference is that in one case the top level directory can contain mails while the other not: 'kmail folders' (broken one) is a directory containing MailDirs, while 'local folders' (working one) is a MailDir... so it can contain nested MailDirs.
2. after the migration you get the broken one.

for some reason i'm unable to use your workaround...
i've logout from kde (just to make sure taht nothing from akonadi is working). then i've removed: ~/.kde/share/config/akondi*, ~/.local/share/akonadi and ~/.config/akonadi. then i've login to kde and 'local folders' were automatically created. i've used akonadiconsole to chenge it's path to ~/.kde/share/apps/kmai/mail but i only get outbox and sent-mail in agents browser under 'local folders'. i;ve tried to restart akonadi server and do synchronization, but nothing happend...

and BTW: is that normal, that my ~/.local/share/akonadi/file_db_data/ is so big? it takes 2,3GB and i see my mails inside it. why do i have to keep all my mails in ~/.kde/share/apps/kmail/mail and in this folder? and a second question - why does it takes 2,3GB if my ~/.kde/share/apps/kmail/mail is 2,9GB?

ok - i've found another bug report with that files in file_db_data, so to make thread clean just ignore my questions...

(In reply to comment #20)
> for some reason i'm unable to use your workaround...
> i've logout from kde (just to make sure taht nothing from akonadi is working).
> then i've removed: ~/.kde/share/config/akondi*, ~/.local/share/akonadi and
> ~/.config/akonadi. then i've login to kde and 'local folders' were
> automatically created. i've used akonadiconsole to chenge it's path to
> ~/.kde/share/apps/kmai/mail but i only get outbox and sent-mail in agents
> browser under 'local folders'. i;ve tried to restart akonadi server and do
> synchronization, but nothing happend...
> 
> and BTW: is that normal, that my ~/.local/share/akonadi/file_db_data/ is so
> big? it takes 2,3GB and i see my mails inside it. why do i have to keep all my
> mails in ~/.kde/share/apps/kmail/mail and in this folder? 

my du -sh ~/.local/share/akonadi/file_db_data/
496K    /home/sergio/.local/share/akonadi/file_db_data/
when 
du -sh ~/.kde/share/apps/kmail/mail 
1.2G    /home/sergio/.kde/share/apps/kmail/mail

> and a second question
> - why does it takes 2,3GB if my ~/.kde/share/apps/kmail/mail is 2,9GB?

for your own good backup ~/.kde/share/apps/kmail/mail is the real folder of your email .

i've faced the same problem with kdepimlibs-4.7.2, so it's not the problem.
i've also noticed in logs:

processing retrieval request for item 19373  parts: ("RFC822", "HEAD")  of resource: "akonadi_mixedmaildir_resource_0" 
"[QSQLITE3: 139855625340672] " sqlite3_blocking_step: Entering while loop 
continuing 
request for item 19373 "1298448716.3654.JTPRF:2,S" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote
application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply,
the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
akonadiconsole(3425)/libakonadi Akonadi::EntityTreeModelPrivate::fetchJobDone: Job error:  "Unknown error. (Unable to fetch item from backend)" 

so i suppose my problem can be related with using sqlite driver. i've read akonadi docs and now i know, that akonadi has known problems with sqlite, so i switched to mysql. it works fine for few days now, so it's possible, that it solves my problem

The problem disappeared for me since I updated the whole KDE from 4.7.3 to 4.7.4 

BTW, it had nothing to do with sqlite, I'm using postgresql

I found a workaround that needs to be issued after each restart, but is much simpler than described above. Every time KMail blocks, I check with akonadiconsole that it is indeed akonadi_mixedmaildir_resource that blocks the process. It tells me it is checking the "outbox" folder but progress is 0% always. I simply do a  

killall akonadi_mixedmaildir_resource

and that's it. However, there is a drawback: I lost some mails in one case since I tried to move mails from one folder to another after the blocking started. By killing as mentioned above, the mails got deleted from the source folder but never arrived at the destination folder. 

Either way I hope very much a bug fix will come out soon, the situation is far from ideal.

I think this report mixes a couple of problems, e.g. comment #2 seems to refer to a problem with the imap resource.

Anyway. I think the main problem has been fixed in 4.7.4 and 4.8, i.e. I could not reproduce the extremely well done test case in comment #9 (indeed thanks a lot Marco).

Since I am still seeing the other problem (the one that generates the error message about missing folder name), I am leaving this report open.

But again I think the problem where it gets stuck has been fixed now. Please add a new comment if you still get it into that state on 4.7.4 or 4.8

Out of the blue on my previously working IMAP connection I'm seeing a bunch of logging like Comment #3 and am unable to access any emails

KMail 4.8 RC2

I can confirm all of the above debug outputs, running KMail 4.7.4 and Akonadi 1.6.2 on Gentoo.

KMail and Akonadi are incredible resource hogs on my system, Akonadi's mysqld takes up to 50 % CPU for long times and KMail blocks and hangs most of the time with Akonadi's constantly pending database requests.

I'm also having a mix of mbox and maildir files of 3,5 GB with about 80.000 e-mails which I migrated automatically when I switched from 4.4 to 4.7.

(In reply to comment #28)
> I can confirm all of the above debug outputs, running KMail 4.7.4 and Akonadi
> 1.6.2 on Gentoo.
> 
> KMail and Akonadi are incredible resource hogs on my system, Akonadi's mysqld
> takes up to 50 % CPU for long times and KMail blocks and hangs most of the time
> with Akonadi's constantly pending database requests.
> 
> I'm also having a mix of mbox and maildir files of 3,5 GB with about 80.000
> e-mails which I migrated automatically when I switched from 4.4 to 4.7.

I just have 1 GB and had to wait 1-2 hours for 3.5 I say 7 hours!.

Once I did a complete regeneration of nepomuk .
Be warned: If you use nepomuk for other data as well you will loose
all this.
Be warned: if you have expired folders that delete permanently you may lose many messages, (just by expire wrong folder ) 
   
I disable nepomuk and stop mysql of akonadi and remove 
rm -rf  ~/.kde/share/apps/nepomuk/repository/*
rm -rf  ~/.local/share/akonadi/db_data/*

enable nepomuk and after start mylsql of akonadi 

and virtuoso-t consumes many CPU 100% of 400% possible . After several
minutes , 1 or 2 hours. Again , mixup filter, expired folders and folder
for Inbox , sent-mails, draft and templates of which id .

This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.

Still happens for me all the time, after nearly 5 (sic!) years. Current Kubuntu, Akonadi 5.1.51.

Example from right now:

processing retrieval request for item 4479  parts: ("RFC822")  of resource: "akonadi_imap_resource_0"
continuing
request for item 4479 "10791" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 4481  there are  1  queues and  0  items in mine
request for item 4481 still pending - waiting
processing retrieval request for item 4481  parts: ("RFC822")  of resource: "akonadi_imap_resource_0"
Database "akonadi" opened using driver "QMYSQL"
continuing
request for item 4481 "10792" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 4481  there are  1  queues and  0  items in mine
request for item 4481 still pending - waiting
processing retrieval request for item 4481  parts: ("RFC822")  of resource: "akonadi_imap_resource_0"
continuing

and so on...

This does not happen with any other email client. So I reckon people have either given up on KMail or just got used to "Work offline"/"Work online". Pretty sad.

Janos, thanks for reporting back. Both bug reporters and developers of KDE PIM are still quite active, although we'd definitely need more developers for this huge project. Since resources are sparse, nobody has the time to look at every bug report, and sure not for bug reports that noone seems to have encountered in a long time.

Anyway, setting this to confirmed.