Bug 248554 - Regression: Imap folder changes are not displayed (no longer synchronized?)
Summary: Regression: Imap folder changes are not displayed (no longer synchronized?)
Status: CLOSED FIXED
Alias: None
Product: KMail Mobile
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Maemo 5 Linux
: VHI critical
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 248787 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-08-20 22:27 UTC by Sabine Faure
Modified: 2010-09-30 00:21 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
No change in Imap resource status displayed on Akonadi console while launching a sync (82.63 KB, image/jpeg)
2010-09-01 18:43 UTC, Sabine Faure
Details
Imap resource still displays 'Ready' as its status while launching a sync (89.59 KB, image/jpeg)
2010-09-01 22:33 UTC, Sabine Faure
Details
Kmail desktop trunk Imap resource syncing correctly (73.93 KB, image/jpeg)
2010-09-01 23:02 UTC, Sabine Faure
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sabine Faure 2010-08-20 22:27:39 UTC
Version:           unspecified (using Devel) 
OS:                Linux

The Imap account is no longer synchronized today whatever I do: the folders are not updated and the emails are not updated either(if some are deleted from another computer they are still displayed on the N900).

Reproducible: Always

Steps to Reproduce:
- Launch Kmail-mobile
- Click on the Imap folder
- Open the 'Actions' tab
- Click on 'Accounts'
- Click on 'Synchronize emails in account'

Actual Results:  
Nothing happens

If you launch an akonadi console on the N900 from another computer you can see that the Imap resource does not sync at all.

Expected Results:  
The Imap resource should sync properly as it was doing yesterday so this is definitely a regression.

N900 Packages, Svn Rev 1165514

DBus timeouts appear in the terminal.
Comment 1 Sabine Faure 2010-08-20 22:29:41 UTC
I am setting this bug as High priority since it prevents me from testing some of the identities functionalities such as sending an encrypted or signed email.

N900 Packages, Svn Rev 1165514
Comment 2 Bernhard E. Reiter 2010-08-25 12:36:12 UTC
I can confirm with n900 tag 20100820-1 that 
new emails are not displayed anymore.
(Sabine: Are you also using a tag? Or if you do not: What packages are you using?)

What is different to Sabine's observations is that
akconadiconsole --remote show that the folder is synced correctly
and has the new emails. They just are not displayed.
(I've triggered the sync by Actions -> Folder -> Sync Emails in Folder)

There is a sister problem with  #242684 -  Mails not received/displayed
https://bugs.kde.org/show_bug.cgi?id=242684 for the desktop, which slightly
behaves differently.
Comment 3 Bernhard E. Reiter 2010-08-25 12:40:31 UTC
*** Bug 248787 has been marked as a duplicate of this bug. ***
Comment 4 Bernhard E. Reiter 2010-08-25 13:08:57 UTC
Maybe a restart can help sometimes. (On one n900 I can now see emails again).
Comment 5 Bernhard E. Reiter 2010-08-25 16:26:32 UTC
Debugging the stuff together with Volker:

a) We saw one problem now after a complete fresh start
using "kmail-mobile.sh" directly. This means akonadiserver was not started
before and gets started by kmail-mobile.sh.

Symptoms: No new emails. Triggering a sync will not have an effect (e.g.
like it normally would in the debug console output).
The imap resource might believe it is offline or online, but it does not connect. Toggling it offline and online might work.

Workaround: start akonadiserver before starting kmail-mobile.sh.
 (usually set the path and then use "akonadictl start")
Check if you can see a successful connection in the debug output.
Then start Mail.

b) A second problem exists when the imap resouces starts to work again, e.g.
by manuall settings: kmail-mobile.sh will not get updates. 
Symptoms: You can see the freshly synced emails in the akonadiconsole, but 
they are not shown in the kmail-mobile.sh interface. Creating new folders
with akonadiconsole will also not lead to display. Thus the connection of kmail-mobile.sh to the backend is disrupted.

Workaround: restart kmail-mobile.sh
Comment 6 Sabine Faure 2010-08-25 19:00:12 UTC
I retested this today and I still have the same bug.

I tried using the workaround but it did not work either (It did sync the local folders and the kolab resource but then I got a timeout dbus: "/tmp/dbus-KfHl7yrcUi" 
"/tmp/dbus-KfHl7yrcUi" 
"/tmp/dbus-KfHl7yrcUi" 
"/tmp/dbus-KfHl7yrcUi".

So I ended up deleting my Imap resource and the Local Folders from akonadi console and reconfiguring it on the N900 and now it seems to work.

N900, 4:4.5~20100824.1167481-1maemo1.1167070
Comment 7 Sabine Faure 2010-08-25 19:20:02 UTC
Even after deleting and recreating the Imap resource after a while the resource stopped responding completely. No idea why (I was trying to remove shared folders when it happened).

This bug is really blocking me. I was not able to test any of the functionality I planned to test today: please correct in priority ;o)

N900, 4:4.5~20100824.1167481-1maemo1.1167070
Comment 8 Volker Krause 2010-08-26 12:02:59 UTC
Revision 1168269 fixes at least parts of this, please let me know if the behavior improved/changed with this.
Comment 9 Bernhard E. Reiter 2010-08-26 12:29:31 UTC
Volker: Backporting to the tag seems useful to me.
Comment 10 Volker Krause 2010-08-26 14:16:49 UTC
SVN commit 1168305 by vkrause:

SVN_MERGE
CCBUG: 248554
Merged revisions 1168259,1168269 via svnmerge from 
svn+ssh://vkrause@svn.kde.org/home/kde/trunk/KDE/kdepimlibs

........
  r1168259 | vkrause | 2010-08-26 11:30:42 +0200 (Thu, 26 Aug 2010) | 3 lines
  
  Make this code slightly less surprising and easier to read, no
  functional changes (yet).
........
  r1168269 | vkrause | 2010-08-26 12:00:57 +0200 (Thu, 26 Aug 2010) | 4 lines
  
  Reconnect the monitor when the server restarts. This is necessary now
  that we switched to the subscription based interface and no longer
  simply listen to broadcasted signals.
........


 _M            . (directory)  
 M  +0 -6      akonadi/changerecorder.cpp  
 M  +23 -5     akonadi/monitor.cpp  
 M  +1 -0      akonadi/monitor.h  
 M  +16 -8     akonadi/monitor_p.cpp  
 M  +4 -1      akonadi/monitor_p.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1168305
Comment 11 Sabine Faure 2010-08-26 23:14:46 UTC
I retested this today and unfortunately the sync still does not work.

Here's what I got in the terminal after clicking on Sync:

akonadi_kolabproxy_resource(3261) KolabProxyResource::retrieveItems: RETRIEVEITEMS
akonadi_kolabproxy_resource(3261) KolabProxyResource::retrieveItemFetchDone: Items is emtpy 
akonadi_kolabproxy_resource(3261) KolabProxyResource::retrieveItems: RETRIEVEITEMS
posting retrieval request for item 24336  there are  0  queues and  0  items in mine 
checking if request for item 24336 has been processed... 
request for item 24336 still pending - waiting 
processing retrieval request for item 24336  parts: ("RFC822")  of resource: "akonadi_imap_resource_9" 
continuing 
checking if request for item 24336 has been processed... 
request for item 24336 processed, error: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 24337  there are  0  queues and  0  items in mine 
checking if request for item 24337 has been processed... 
request for item 24337 still pending - waiting 



N900, 4:4.5~20100826.1168354-1maemo1.1168271
Comment 12 Volker Krause 2010-08-27 15:05:33 UTC
SVN commit 1168799 by vkrause:

Disable conflict detection in ItemSync for now.

We currently have no way yet to handle LR conflicts, and there are cases
where this can actually dead-lock the ItemSync/TransactionSequence
combination.

BUG: 248554


 M  +1 -0      itemsync.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1168799
Comment 13 Sabine Faure 2010-08-27 21:53:12 UTC
I retested this today and it still does not sync unfortunately.

I get the same info as in comment #11 in the terminal (see above) so I am reopening this bug.


N900,4:4.5~20100827.1168874-1maemo1.1168783
Comment 14 Bernhard E. Reiter 2010-08-31 09:22:14 UTC
Initial sync at startup was already better for me with 20100820-2 (backported
fix). I have not done an intensive test, though.
Also Sabine and Björn Balazs still reported sync problems.
Comment 15 Bernhard E. Reiter 2010-08-31 09:50:18 UTC
First impression is better with the new tag:
N900 kdepim-mobile         4:4.5~20100827.1168757-1maemo1.1169941
libqt4-experimental-gui    4.7.0~git20100824-0maemo1

I could immedeately see some new emails in a subfolder.
Comment 16 Volker Krause 2010-08-31 10:16:49 UTC
Please check with akonadiconsole if the affected resource is marked as offline when this happens, and check if switching it back to online fixes it (it should stay online then of course).
Comment 17 Manuel Nickschas 2010-09-01 12:39:27 UTC
I can reproduce this with *desktop* kdepim (current trunk, Linux, amd64). Used to work maybe a week ago, now I can't sync anything anymore, getting the aforementioned dbus errors.

The resource is not marked as offline, and I can see all my folders (but not access any mail, as downloading a mail doesn't work either).

So the product should maybe be more generic.
Comment 18 Sabine Faure 2010-09-01 18:43:23 UTC
Created attachment 51184 [details]
No change in Imap resource status displayed on Akonadi console while launching a sync

I retested the Imap not synchronizing bug today and it still behave in exactly the same way as before. 
I do not see any change after the correction unfortunately.

There is no ready or offline status displayed (see Akonadi console snapshot for details).

I've also noticed that 'Local Folders' has lost its name, it is now called Maildir. 
None of the resources in Akonadi console have a ready status anymore and none of them are synchronized anymore (whereas last week 'Local Folders' and the Kolab resource where synchronizing correctly when clicking on Actions tab - Home - Synchronize all emails) See Snapshot for details.

N900, 4:4.5~20100901.1170517-1maemo1.1170522
Comment 19 Sabine Faure 2010-09-01 18:50:53 UTC
Volker, I've tried making the Imap resource synchronize from the Akonadi console (Right click on the resource - Synchronize/Synchronize all) but it never worked fo me. Nothing happens.

N900, 4:4.5~20100901.1170517-1maemo1.1170522
Comment 20 Sabine Faure 2010-09-01 22:33:46 UTC
Created attachment 51195 [details]
Imap resource still displays 'Ready' as its status while launching a sync

I retested this with a more recent revision and now I get the same result as last week:

only the Imap resource refuses to sync now whether the sync is activated from the N900 or Akonadi console (see snapshot for details).

N900, 4:4.5~20100901.1170574-1maemo1.1170522
Comment 21 Sabine Faure 2010-09-01 22:56:28 UTC
Following comment #17 I retested Kmail desktop trunk and the synchronization works fine for me (see snapshot enclosed).

I was able to send emails from the N900 and read them using Kmail desktop trunk.
I could also launch a synchronization of Kmail desktop from Akonadi console successfully.

Manuel, please retest it should work for you now.

Trunk, Svn Rev 1170619
Comment 22 Sabine Faure 2010-09-01 23:02:07 UTC
Created attachment 51198 [details]
Kmail desktop trunk Imap resource syncing correctly 

Trunk, Svn Rev 1170619
Comment 23 Sabine Faure 2010-09-08 18:43:44 UTC
It is corrected now.

This does not happen anymore so I am verifying that bug.

N900, 4:4.5~20100906.1172093-1maemo1.1171830
Comment 24 Sabine Faure 2010-09-09 19:42:41 UTC
I retested this today and this is still happening so I am reopening this bug.

Here is more information on what is happening:

1. I deleted my previous Imap resource and configured a new one that worked fine all day yesterday.

2. Today, however when I relaunched Kmail-mobile there was no automatic first sync (Could that be because I set the automatic sync interval to 120 minutes?). The resources status in Akonadi console were displayed as 'Ready'.

3. So I launched a manual sync from the N900. I could see that the Local Folders were correctly synchronized. But there is still the same pb with Kolab (I get the exact same errors as in comment #11 in the terminal) and it does not seem to Sync properly: it stays forever in each folders in Akonadi console though it manages to move from one to the next.

4. After a long long while (about 1 hour) nothing more happens, The IMAP resource is never synchronized not even when the user specifically asks to sync the Inbox Imap from the N900 or when its sync is requested from Akonadi console (right click on Imap resource - synchronize/Synchronize All). Every resources in Akonadi console is now displayed with a plug instead of 'Ready' for status. 

I've tried stopping and restarting akonadi server and then relaunching Kmail-mobile but it does not work (It does make the resources status switch back to 'Ready' but there are still no automatic sync on start up and no manual one works either.

It seems that nothing else than deleting and reconfiguring the Imap resource makes it synchronize again atm but that's only a work around...

N900, 4:4.5~20100909.1173240-1maemo1.1173275
Comment 25 Felix Wolfsteller 2010-09-21 13:56:24 UTC
I have seen similar behavior in the desktop client and have the impression that on the mobile device this behavior sometimes occurs after a certain amount of time. I cannot find a reproduceable setting or a pattern, though.

Also, sync feedback will make this much more easy to observe (reported as another, high prioritized issue).
Comment 26 Sabine Faure 2010-09-23 18:37:53 UTC
I retested this bug several time in a row today and syncing works fine now so I am verifying this bug.

According to ervin this bug may have been linked to a subscription bug that is now fixed.

N900, 4:4.5~20100922.1178352-1maemo1.1177763
Comment 27 Sabine Faure 2010-09-30 00:21:01 UTC
I retested this bug again today and it still works fine so I am closing it for good ;o)

N900, 4:4.5~20100929.1180953-1maemo1.1180919 



Note: when it was not working I was able to reproduce it that way:

'Imap resource does not sync huge shared folders properly'

When the serverside subscription is disabled and the user syncs the Imap account, there are dbus errors while syncing huge folders.

- Launch kmail-mobile
- Click on your imap account
- Open the 'Actions' tab
- Click on 'Account'
- Click on 'Edit account'
- Uncheck 'Enable Server-Side subscriptions
- Click on 'Ok'
- Sync the Imap account if it does not sync automatically (it should)


The sync works fine until it reaches a huge folder then dbus errors are displayed in the terminal.

When it happens the Imap resource get stuck and does not respond anymore.

Huge folders should be sync correctly and Imap resource should not get stuck.

Note: the folder view is updated correctly and displays the shared folders names after disableing the serverside subscription and a sync.