Bug 351628

Summary: Akonadi DAV Resource does not sync all calendars on ownCloud any more
Product: [Frameworks and Libraries] Akonadi Reporter: Matija Šuklje <matija>
Component: DAV ResourceAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED REMIND    
Severity: critical CC: bugs.kde.org, greg, kdebugs_jm, nickbryda, randy, stamper
Priority: NOR    
Version: 1.13.0   
Target Milestone: ---   
Platform: Mageia RPMs   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: Akonadi Console ⇒ Debugger tab ⇒ Dav Groupware tab
Akonadi Console ⇒ Job Tracker tab
console output of `akonadictl restart` (only CalDav related)
ownCloud log from the same time
ownCloud log from around the same time
Debug Log for akonadi console

Description Matija Šuklje 2015-08-22 21:09:00 UTC
What happens is that every now and again, one or several of the calendars (and/or contacts) resources on ownCloud stop syncing with Akonadi. Since the sync works fine with my Jolla phone, I suspect the issue is on KDE’s side.

How this looks in practice is that suddenly and without warning items from some calendars are missing in KOrganizer, Zanshin, EventList and even Akonadi Console.

What I already tried to do, that before worked, but since the latest update doesn’t:
 • in Akonadi Console: toggle offline; abort activity; restart resource; toggle online
 • in Akonadi Console: restart Akonadi server
 • in Akonadi Console: clone the WebDAV resource and remove the old one
 • in Akonadi Console: just remove the old one and create a new one manually
 • restarting KDE/X
 • restarting the computer (yes, I’m getting desperate!)
 • created a fresh new user on my laptop with a clean new home and added my ownCloud as a new CalDAV resource
 • export the iCal file from ownCloud and reimport it to ownCloud, so I have a duplicate (funnily enough if I open that iCal file in KOrganizer it works)

 …all to no avail. The events don’t show in the Akonadi Console.

The current state that I can see in Akonadi Console ↦ Browser is that:
 • Contacts shows 673 entries (seems correct)
 • “Dogodki” calendar shows 17 entries (seems correct)
 • “Faks” calendar shows no entries and 0 size (**FALSE**)
 • “FSFE” calendar shows 579 entries (seems correct)
 • “Osebno” calendar shows no entries and 0 size (**FALSE!**)
 • Rojstni dan/Birthdays show 14 entries

The only thing that I can think of as special for the two non-syncing calendars is that I renamed them at some point (Študij ⇒ Faks; Opravki ⇒ Osebno) and in the backend they are still called the old name. But this was long long before this bug happened and it works fine with the Jolla Sailfish calendar app.

My setup: Mageia 5 (stable); x86_64
Akonadi 1.13.0-4
KDE: 4.14.5
KOrganizer: 4.14.5-1
ownCloud: 8.0.4

I suspect this could be related to: https://bugs.kde.org/show_bug.cgi?id=328734

Confirmed at Akademy, when dvratil (will add to CC) looked at my laptop and tried to figure out what’s the cause.

Reproducible: Always
Comment 1 Matija Šuklje 2015-08-22 21:10:22 UTC
Created attachment 94177 [details]
Akonadi Console ⇒ Debugger tab ⇒ Dav Groupware tab
Comment 2 Matija Šuklje 2015-08-22 21:11:03 UTC
Created attachment 94178 [details]
Akonadi Console ⇒ Job Tracker tab
Comment 3 Matija Šuklje 2015-08-22 21:11:59 UTC
Created attachment 94179 [details]
console output of `akonadictl restart` (only CalDav related)
Comment 4 Matija Šuklje 2015-08-23 00:40:10 UTC
Currently I’m trying this workaround:

1) in ownCloud export calendar to .ics
2) add .ics as Local iCal file in Akonadi/KOrganizer
3) create a new (empty) calendar on ownCloud
4) copy all events and todos from said Local iCal to the new calendar (e.g. via Akonadi Console)
5) remove the old calendar on ownCloud

After a restart, it seems that it works, but it’s too early to tell yet how long that will last. For starters, now the biggest calendar I have instead of containing 1016 items, only contains 202 items.
Comment 5 Matija Šuklje 2015-08-25 09:12:12 UTC
The only odd thing that I found in said calendar was that in one of the events there was a newline in the title. I removed the newline just in case in step 4) from my previous comment.

Other then that, I don’t see any oddness.
Comment 6 Grégory Oestreicher 2015-08-30 09:27:09 UTC
Could it be that the calendars that are empty all contain events that are more than 3 months old?
Comment 7 Matija Šuklje 2015-08-30 13:33:12 UTC
(In reply to Grégory Oestreicher from comment #6)
> Could it be that the calendars that are empty all contain events that are
> more than 3 months old?

No.
Comment 8 Grégory Oestreicher 2015-08-30 18:22:28 UTC
The logs only show that when the resource tries to synchronize https://thatfunkyplace.wheremymonkeyis.at/remote.php/caldav/calendars/hook/Študij/ and https://thatfunkyplace.wheremymonkeyis.at/remote.php/caldav/calendars/hook/opravila/ it doesn't find anything.

Do you have something in the ownCloud logs or the Apache logs? Any error?
Comment 9 Matija Šuklje 2015-09-01 15:43:05 UTC
(In reply to Matija Šuklje from comment #4)
> Currently I’m trying this workaround:
> 
> 1) in ownCloud export calendar to .ics
> 2) add .ics as Local iCal file in Akonadi/KOrganizer
> 3) create a new (empty) calendar on ownCloud
> 4) copy all events and todos from said Local iCal to the new calendar (e.g.
> via Akonadi Console)
> 5) remove the old calendar on ownCloud
> 
> After a restart, it seems that it works, but it’s too early to tell yet how
> long that will last. For starters, now the biggest calendar I have instead
> of containing 1016 items, only contains 202 items.

This workaround seems to have deleted the items on my ownCloud as well now, so I only have ~200 items instead of ~1000 in it. I suspect that this part is due to the 3 month issue.

I haven’t looked at the ownCloud and Nginx logs yet. Will do that later, hopefully today.
Comment 10 Matija Šuklje 2015-09-01 20:32:36 UTC
Created attachment 94333 [details]
ownCloud log from the same time

Sadly, I don’t have the Nginx logs from that far back in time, but at least I could find the ownCloud logs with the same timestamp.
Comment 11 Matija Šuklje 2015-09-01 20:51:03 UTC
Created attachment 94336 [details]
ownCloud log from around the same time
Comment 12 Grégory Oestreicher 2015-09-01 21:03:56 UTC
Thanks for the logs. I'll have to whip up a script to make them a bit more readable, because they're… wow… hard to read would be an understatement :)
Comment 13 Matija Šuklje 2015-09-01 21:17:14 UTC
Thanks for tackling this :)

I tried to help out by just including 2-3 hours around the time when I recorded the Akonadi logs as well. It’s actually not many entries – just stupidly long lines.
Comment 14 Grégory Oestreicher 2015-09-07 20:16:07 UTC
Well, I'm a bit stuck here. It seems that ownCloud chokes on a malformed date time. As this happens only for the 'opravila' calendar I'm wondering if there isn't a malformed event in it somewhere.

If you have access to the owncloud database with a SQL client can you look if you have an event whose start time would be equal to '-4713010'? Assuming your table prefix is the default 'oc_' the following request may find it: "SELECT * FROM oc_clndr_objects WHERE calendardata LIKE '%-4713010%';".

This doesn't explain why the other calendar doesn't sync, but there's nothing about this one in the logs :(
Comment 15 Joerg Mertin 2015-09-10 08:10:54 UTC
Hi folks,

it does not seem to be linked solely to Kontact/Akonadi on the KDE side.
I have the same effect at home (using latest owncloud 8.1.1). 

I'm connected using latest Kontact ruinning KUbuntu 15.04  using KDE Frameworks 5.12.0.
My wife however is using Mac OS-X 10.6.x - and the OS-X Calendar application of that version.

Thing is - both these systems show the described behavior in this bug-report.

Only on her iPad mini and on my Android devices - do we not see this error.
Comment 16 Grégory Oestreicher 2016-02-09 20:40:32 UTC
I know this is a bit old, but do you still have this issue? If so could you try send me anonymized ownCloud logs to see if this is caused by an invalid item?
Comment 17 somnium 2016-03-16 14:09:30 UTC
I'm not quite sure, if this is the same problem, but my owncloud contacts and calendars are not shown in kontact etc. anymore. In the akonadi console the resources are present and "ready", so no obvious errors. But there is no entry in kontact.

I'm using owncloud 9.0 and php7 on the server, and Arch linux with the KDE Applications 15.12.3 on the client side. The calenders and contacts are shown in android and ios, as well as thunderbird.
Comment 18 somnium 2016-03-16 14:22:03 UTC
I've just recheck the akonadi console and found, that there is no data in the caldav-resource (0 kb). So it seems, the data is not synced from the server.
Comment 19 somnium 2016-03-17 22:13:25 UTC
Created attachment 97951 [details]
Debug Log for akonadi console

Debug log from Akonadi-Console after the update of the owncloud-resource
Comment 20 somnium 2016-04-02 22:55:16 UTC
It seems that the problem has something to do with the new owncloud version 9.0.

The urls changed from: https://ADDRESS/remote.php/caldav/ and https://ADDRESS/remote.php/carddav/ to https://ADDRESS/remote.php/dav/ (caldav and carddav). I changed this manually in akonadiconsole and the calendars and contacts appeared again.

I think, this is an easy fix in kontact/kdepim.

Cheers.
Comment 21 Erik Quaeghebeur 2018-11-16 12:15:00 UTC
Is this still an issue?
Comment 22 Matija Šuklje 2018-11-16 13:24:19 UTC
I’ll keep an eye open. I knew I had some out of sync events recently, but can’t recall if it was via Nextcloud (14) or Google Calendar.
Comment 23 Bug Janitor Service 2018-12-01 03:44:41 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 24 Matija Šuklje 2018-12-05 10:35:27 UTC
This is a bug that does not happen on a regular basis or very often.

I would suggest either prolonging the NEEDSINFO time for let’s say 2 months.
Comment 25 Erik Quaeghebeur 2018-12-05 11:34:29 UTC
Marking this as RESOLVED REMIND instead. This means that if the bug reappears in a month, a year, whatever, you can comment ask for it to be reopened. But please check related open bugs at that time to avoid duplication <https://bugs.kde.org/buglist.cgi?component=DAV%20Resource&product=Akonadi&resolution=--->. (You'll have to copy/paste the link, as the last part is not detected as part of it.).
Comment 26 Matija Šuklje 2018-12-05 12:04:47 UTC
(In reply to Erik Quaeghebeur from comment #25)
> Marking this as RESOLVED REMIND instead. This means that if the bug
> reappears in a month, a year, whatever, you can comment ask for it to be
> reopened.

Perfect! Thanks for understanding :)
Comment 27 Matija Šuklje 2018-12-14 10:43:07 UTC
I just deleted and re-created the CalDAV account, as it did not sync some invitation that I got from Uber Conferences.

I got the invitation by e-mail as a VCS, which I opened in KMail, which opened KOrganizer, where I imported it to the CalDAV calendar. It didn’t show in KOrganizer, but it shows on Nextcloud (my CalDAV server).

Removing and re-creating the account in Akonadi did not solve it though, so I guess this is a different issue. Still wanted to update you on this.
Comment 28 Erik Quaeghebeur 2018-12-14 11:00:33 UTC
(In reply to Matija Šuklje from comment #27)
> I got the invitation by e-mail as a VCS, which I opened in KMail, which
> opened KOrganizer, where I imported it to the CalDAV calendar. It didn’t
> show in KOrganizer, but it shows on Nextcloud (my CalDAV server).
> 
> I guess this is a different issue. Still wanted to update you on this.
This sounds like a different issue. Just to check: is the calendar actually selected in KOrganizer? (In the bottom-left tree view of calendars.) If it is, check the KOrganizer bugs to see if one fits or otherwise file a new one.
Comment 29 Matija Šuklje 2019-01-04 11:31:28 UTC
It is selected, yes.

I think the issue with invites not showing is that the “Search” resource does not seem to work – at least, I never saw anything in either the “Open invitations” or the “Declined invitations” ‘calendar’.