Bug 412722

Summary: KOrganizer does not show some events from EWS resource
Product: [Applications] korganizer Reporter: Martin Höher <martin>
Component: generalAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: auxsvr, dag, dvratil, julien.montant, kde, leszek.lesner, marcin.bajor, mullein, philipp.reichmuth, winter
Priority: NOR    
Version: 5.11.2   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: 07.02.2023 Comparing events from widget and Korgranizer
Event in akonadi
Akonadi payload

Description Martin Höher 2019-10-08 08:30:08 UTC
SUMMARY

Events from an EWS resource are not shown in KOrganizer.


STEPS TO REPRODUCE
1. Set up an EWS resource
2. Make sure, e-mails show up in KMail.

OBSERVED RESULT

In KOrganizer, no events are shown.


EXPECTED RESULT

KOrganizer should show events synced via EWS.


SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: Fedora 30, Kontact 5.11.2
(available in About System)
KDE Plasma Version: 5.15.5
KDE Frameworks Version: 5.59.0
Qt Version: 5.12.4

ADDITIONAL INFORMATION

The EWS sync for mails works properly, i.e. I can see and write mails via KMail. The calendar folders are also showing up, however, there are no events shown in neither view of KOrganizer. The most interesting part however is, that the events are shown in the calendar plasmoid via the PIM events plugin. So it seems, internally Akonadi syncs these items. They are "just not drawn" by KOrganizer...
Comment 1 Martin Höher 2019-10-08 11:59:32 UTC
Btw: I just realized than when scrolling back a bit in the calendar into past months, I see some (but not all) events I had. So KOrganizer internally seems to see at least a subset of the items available on the server and probably fetched by Akonadi.
Comment 2 Martin Höher 2019-10-08 15:49:04 UTC
Another observation: It seems that KOrganizer renders events which I created myself (via the Office 365 web interface). But ones created by others (where I am just invited) are not shown.
Comment 3 Allen Winter 2019-10-08 16:17:02 UTC
if you could find the "invisible" events in another calendar program and email them to me that would help.  Don't send me any events with personal info, or figure out how to remove the personal info.

it's possible the events are formatted in a way that korganizer doesn't like yet.
Comment 4 Allen Winter 2019-10-09 14:05:07 UTC
Martin sent me a test .ics file.
It has double-spacing and does not use CRLF lineending.

Indeed KOrganizer claims to have imported that file but the event is not shown.
I removed the double-spacing and converted it to have CRLF lineendings
and it imports fine and I can see the Testing event.

I ran the original .ics file in the validator it shows problems about
the CRLF lineendings and also warns about the double-spacing

So definitely not a "valid" ics file.
Next question is where did it get messed up?

Can't be Akonadi itself.  Could it be the EWS resource?
Comment 5 Julien 2020-11-07 15:14:59 UTC
Are there any news here?

I'm experiencing exactly the same issue, using KOrganizer with davmail.
As for Martin Höher, everything works properly, except events created by other users (that I accepted). Note that exactly the same davmail setting using Thunderbird Lightning as a client works perfectly, so that's definitely related to akonadi/KOrganizer.

My configuration is the following:
- Archlinux with kernel 5.9.4-arch1-1
- Korganizer 5.15.3 (20.08.3)
- Davmail 5.5.1
- The EWS resource is an Office 365 account.

I'm not used to debugging KDE applications, but I would be very happy to help in any way if you can give me some indications.
Comment 6 Martin Höher 2022-07-05 09:46:50 UTC
Sorry to spam here, but: Any updates on the topics?

I just checked again with the latest KOrganizer installed from Fedora repositories (it shows version 5.19.2 (21.12.2) in the about dialog.

I still see the same: Events created by myself seem to work but not ones I've been invited to. Interestingly, all of the events properly show up in akonadiconsole, too - including start/end time, message body, ... Could that help debugging the issue further?
Comment 7 Martin Höher 2022-07-05 11:00:29 UTC
And another observation:

It seems that this issue only occurs with an Office 365 account. On work, we have two accounts, an O365 one (with that one I have this problem) and an in-house Exchange server (and here - it seems - all events are properly showing up).

So, could this be due to the server version (O365 is running always on latest greatest while the in house server is on Exchange 2016 version).
Comment 8 Martin Höher 2022-07-05 11:15:58 UTC
I'm really sorry and promise I will stop spamming. But just in case this helps:

I just found kalendar (https://github.com/KDE/kalendar) and tried it - but it has the same issues.
Comment 9 Aaron 2022-12-02 16:28:52 UTC
I seem to have a similar issue, or maybe the same.  In my case, I can usually see calendar items that are added by others using Outlook (Exchange hosted on https://outlook.office365.com).  Events that fail to show up on my calendar seem to be events that have some how been updated by the event organizer.  This is even true if I'm the event organizer, and is true for even minor changes (like event name).  Restarting akonadi causes these events to disappear completely.  

So, to reproduce... 
1. Create an event from Outlook/Office365 (the event is synchronized to Korganizer without issue).
2. From Office365, change the title of the event (or recurrence, etc).
3. Restart akonadi and resynchronize Korganizer and watch the event disappear.

For what it's worth, Evolution doesn't seem to have this problem.

Please let me know if I can provide more information or help with this issue.
Comment 10 Aaron 2022-12-02 17:30:27 UTC
Actually...maybe I spoke too soon.  While the behavior I described above is still accurate and reproducible, it does seem that events organized by others are inconsistently synchronized to Korganizer.  I would say, most such events are not synchronized (although they do appear in Akonadi Console).  

I also tried moving an event from my Exchange calendar to a CalDav calendar using Evolution, and that CalDav event is also not synchronized to KOrganizer either (although it is visible from other clients).  So, I guess I'm just further confirming the issue others have raised.
Comment 11 Marcin Bajor 2023-02-09 13:15:35 UTC
I observe the same issue with EWS and CalDav from Davmail on the same account.
I had to delete some confidential data from the files below.
Comment 12 Marcin Bajor 2023-02-09 13:17:48 UTC
Created attachment 156100 [details]
07.02.2023 Comparing events from widget and Korgranizer

I have 4 events 07.02.2023. Note that the widget in the right up corner displays all 4 events.
Comment 13 Marcin Bajor 2023-02-09 13:24:27 UTC
Created attachment 156101 [details]
Event in akonadi

Note that akonadi imports all events from davmail
Comment 14 Marcin Bajor 2023-02-09 13:38:18 UTC
Created attachment 156102 [details]
Akonadi payload

Akonadi payload. I had to change some sensitive data with CONFIDENTIAL word. I hope this will help.
Comment 15 dag 2023-11-14 11:08:51 UTC
I think this is the same bug as the one I see: https://bugs.kde.org/show_bug.cgi?id=418811 . The entry is there, but not shown for some reason.
Comment 16 Jan Bundesmann 2023-12-15 07:46:40 UTC
This might be the same bug that's bothering me: https://bugs.kde.org/show_bug.cgi?id=421916
Comment 17 Leszek Lesner 2024-01-30 11:32:08 UTC
This patch applying to akonadi-calendar fixes it for me. Why those blocked things cannot be configured is a mystery to me.

diff --git a/src/etmcalendar.cpp b/src/etmcalendar.cpp
index f662541616cba2651c742038b43f405404802d21..af973fc6ee7f5c6e2a2d278ad44a5fa1612b57ab 100644
--- a/src/etmcalendar.cpp
+++ b/src/etmcalendar.cpp
@@ -154,7 +154,6 @@ void ETMCalendarPrivate::setupFilteredETM()
     mCalFilterPartStatusProxyModel = new CalFilterPartStatusProxyModel(this);
     mCalFilterPartStatusProxyModel->setFilterVirtual(false);
     QList<KCalendarCore::Attendee::PartStat> blockedStatusList;
-    blockedStatusList << KCalendarCore::Attendee::NeedsAction;
     blockedStatusList << KCalendarCore::Attendee::Declined;
     mCalFilterPartStatusProxyModel->setDynamicSortFilter(true);
     mCalFilterPartStatusProxyModel->setBlockedStatusList(blockedStatusList);
Comment 18 dag 2024-02-12 13:32:44 UTC
Seems to solve the problem, thank you very much!

And completely agree with you that you should be able to select what you want to exclude from the visible events!
Comment 19 dag 2024-02-12 13:40:11 UTC
Well it helped "somewhat". Most of the events are now visible - But not all. Still missing some and they are only invisible in the Month view. Clicking "Agenda" I can see them,
Comment 20 Leszek Lesner 2024-02-12 13:42:17 UTC
Maybe the other blockedStatusList for declined atendees is blocking it. Try removing that as well and see if it fixes it for you.
Comment 21 dag 2024-02-13 14:16:56 UTC
Tried that and it did not help.
Copy/pasted the event from akonadi-console (Why is there no option to export an event to an ics file btw. ?) , edited and re-imported.

When I removed the following line it showed up on the month view as well:

RECURRENCE-ID;TZID=Europe/Vilnius;RANGE=THISANDFUTURE:20240220T150000

This is generated by Teams and akonadi didn't like it.

1. Who has made the decision that a "broken" event will not be displayed at all? It should of course be displayed, perhaps with a marker. The same with an non-accepted event. Not displaying these is a disaster in a serious business environment.
2. What is wrong with the line?
Comment 22 Daniel Vrátil 2024-02-16 10:32:35 UTC
So, here's an answer on what's going on:

Some years ago, special "Search" -> "Open Invitations" and "Declined Invitations" calendars were introduced. Those calendar should contain all your open ("unanswered") and declined invitations from all calendars. To prevent such events appearing twice in KOrganizer, an additional filter was added to code that hides those events from regular calendars. The problem is that users often don't have those special search calendars enabled, and that the special calendars don't work reliably at all and thus events might be missing.

I have accidentally fixed this in git master (to be released 24.02) by porting event views away from ETMCalendar, so this filter was also removed.

I have proposed to remove the special search collections completely (due to their unreliability as of now), always show unanswered invites  make it globally configurable in KOrganizer whether to show declined event.
Comment 23 Daniel Vrátil 2024-02-21 07:58:49 UTC
I'm closing this as fixed, since the original issue has been fixed - open and declined invitations will appear in calendars again in 24.02 release. Current KOrganizer master (to be released in 24.05) has additionally a checkbox in settings to globally show/hide declined invitations.

Dag, it looks like your problem might be caused by some other issue, please open a separate bug report and please attach the problematic event (stripped of PII). Thanks
Comment 24 dag 2024-02-26 08:48:33 UTC
Thanks for the explanation. And I would welcome a removal very much! Better see them twice than not at all (and miss the event)