Bug 401054 - Stream parser exceptions
Summary: Stream parser exceptions
Status: REPORTED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: IMAP resource (show other bugs)
Version: 5.17.80
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 440345 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-11-15 02:58 UTC by Aaron Williams
Modified: 2021-12-06 22:01 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron Williams 2018-11-15 02:58:54 UTC
When I run akonadictl start my console will start flooding with the following message:

org.kde.pim.kimap: The stream parser raised an exception: Unable to read more data
org.kde.pim.imapresource: Append failed:  "Connection to server lost."

I also do not seem to be receiving any new email when this happens.  The mail server is an Exchange server through Microsoft's Office365 service.

I have a second account on my home Cyrus IMAP server. Most of the problems seem to be with the Microsoft server.

I have tried akonadictl fsck and akonadictl vacuum to no effect. Out of desparation I have deleted all of the databases and subdirectories under .local/share/akonadi, leaving just the configuration files.

I also have Akonadi configured to use Postgresql.  This system has been working fine for several years, though I've frequently had the issue where I need to restart Akonadi in order to download new emails.

Akonadi version: 17.12.3

SOFTWARE/OS VERSIONS
Windows: 
MacOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 5.12.6
KDE Frameworks Version: 
Qt Version: 5.9.4

Kontact: 5.7.3
Comment 1 Aaron Williams 2018-11-15 03:01:33 UTC
Note that this is the version included with Opensuse 15.0

> akonadictl status
Akonadi Control: running
Akonadi Server: running
Akonadi Server Search Support: available (Remote Search)
Available Agent Types: akonadi_akonotes_resource, akonadi_archivemail_agent, akonadi_birthdays_resource, akonadi_contacts_resource, akonadi_davgroupware_resource, akonadi_ews_resource, akonadi_ewsmta_resource, akonadi_facebook_resource, akonadi_followupreminder_agent, akonadi_googlecalendar_resource, akonadi_googlecontacts_resource, akonadi_ical_resource, akonadi_icaldir_resource, akonadi_imap_resource, akonadi_indexing_agent, akonadi_invitations_agent, akonadi_kalarm_dir_resource, akonadi_kalarm_resource, akonadi_knut_resource, akonadi_kolab_resource, akonadi_maildir_resource, akonadi_maildispatcher_agent, akonadi_mailfilter_agent, akonadi_mbox_resource, akonadi_migration_agent, akonadi_mixedmaildir_resource, akonadi_newmailnotifier_agent, akonadi_notes_agent, akonadi_notes_resource, akonadi_openxchange_resource, akonadi_pop3_resource, akonadi_sendlater_agent, akonadi_tomboynotes_resource, akonadi_vcard_resource, akonadi_vcarddir_resource

Here is the verbose output I am seeing:

org.kde.pim.akonadiserver: processing retrieval request for item (3483523, 3483582, 3483615, 3483619, 3483630, 3483633, 3483635, 3483657, 3483660, 3483676, 3483681, 3483685, 3483701, 3483703, 3483705, 3483707, 3483709, 3483715, 3483719, 3483730, 3483738, 3483739, 3483756, 3483759, 3483827, 3483844, 3483856, 3483857, 3483881, 3483928, 3483935, 3483946, 3483969, 3483994, 3483997, 3484027, 3484028, 3484032, 3484038, 3484050, 3484072, 3484080, 3484098, 3484104, 3484111, 3484209, 3484263, 3484307, 3484336, 3484354, 3484369, 3484393, 3484399, 3484438, 3484452, 3484466, 3484469, 3484493, 3484538, 3484616, 3484715, 3484758, 3484767, 3484846, 3484867, 3484917, 3484918, 3485055, 3485070, 3485078, 3485080, 3485084, 3485107, 3485124, 3485148, 3485248, 3485279, 3485432, 3485455, 3485461, 3485463, 3485476, 3485477, 3485480, 3485503, 3485509, 3485518, 3485524, 3485539, 3485541, 3485546, 3485550, 3485564, 3485570, 3485572, 3485574, 3485580, 3485607, 3485608, 3485617, 3485635)  parts: ("RFC822", "HEAD")  of resource: "akonadi_akonotes_resource_0"
org.kde.pim.kimap: The stream parser raised an exception: Unable to read more data
org.kde.pim.kimap: The stream parser raised an exception: Unable to read more data
org.kde.pim.akonadiserver: ItemRetrievalJob finished for request 0x7f1608221d90 , error: ""
Comment 2 Aaron Williams 2018-11-15 03:06:31 UTC
In the Mail accounts page it keeps flicking between Ready (0%) and briefly flashing other stuff so fast I can't read it (something established.) This is with the Office365 account. My other account with Cyrus rarely, if ever, has any problems.
Comment 3 David Faure 2019-10-22 14:57:54 UTC
This error (which comes from the function below in kimap) means that no data came onto the IMAP socket for 30 seconds. I guess this could happen with very slow networks?


bool ImapStreamParser::waitForMoreData(bool wait)
{
    if (wait) {
        if (m_socket->bytesAvailable() > 0 ||
                m_socket->waitForReadyRead(30000)) {
            m_data.append(m_socket->readAll());
        } else {                                                                                                                                                                                                                           
            return false;
        }
    }
    return true;
}
Comment 4 David Faure 2019-10-24 10:29:40 UTC
Well, either that or the socket got closed meanwhile. I managed to reproduce this a few times so I added a QElapsedTimer and 0 ms had elapsed. So it wasn't a timeout, but a closed socket, very likely.
Comment 5 Aaron Williams 2019-10-24 20:52:36 UTC
I believe this is due to https://bugs.kde.org/show_bug.cgi?id=403903
Comment 6 Aaron Williams 2021-06-07 21:38:58 UTC
I am seeing this happen again. It now happens immediately with Microsoft office365.com. There is no delay.
Comment 7 Aaron Williams 2021-08-10 16:58:46 UTC
This happens every time I receive a corporate-wide email with an extremely long CC list (the cc line is over 15,000 characters in length). Once this email is hit no more emails can be downloaded. Usually, this is an invitation to a corporate-wide meeting in my case. The only way to recover is to:
1. Stop akonadi
2. Delete the calendar entry in the Office365 web GUI
3. Delete the email in my inbox in the web GUI
4. Delete it from the deleted items folder in the web gui
5. Make sure it's really gone
6. Start akonadi
7. Sometimes I can use akonadiconsole to delete the offending email and allow akonadi to work again, but this isn't reliable.
8. If 7 doesn't work:
 9. stop akonadi
 10. delete .local/share/akonadi/file_db_data and .local/share/akonadictl/search_db
 11. start akonadi
 12. run akonadictl fsck

Once this is done things will work again, however it is a very slow process for Akonadi to rebuild everything since in my case I have several IMAP accounts with decades of emails. Also, I have to repeat this process every single time I receive a corporate-wide email or calendar invitation and it means I can't add it to my calendar.
I appologize for the two duplicate bugs I entered but they might add some additional info not listed here.

I consider this a critical bug, since this basically kills Akonadi every time an email with a huge CC list is received where recovery is difficult.
Comment 8 Aaron Williams 2021-08-10 16:59:42 UTC
*** Bug 440345 has been marked as a duplicate of this bug. ***
Comment 9 David Faure 2021-08-23 15:39:00 UTC
You could try increasing the 30000 in 
bool ImapStreamParser::waitForMoreData (akonadi git repository), just to see if this 30s timeout is the reason for the problem? Maybe the server takes a long time to prepare sending this line...
Comment 10 Aaron Williams 2021-08-25 06:12:47 UTC
I have seen values of 5 minutes or even longer being returned by the server.