Bug 285743 - recurring events in Kolab Storage Format v2 (created with old Kontact e35 versions) display at wrong time in either summer- or winter time
Summary: recurring events in Kolab Storage Format v2 (created with old Kontact e35 ver...
Status: RESOLVED UNMAINTAINED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: Kolab Resource (show other bugs)
Version: GIT (master)
Platform: Ubuntu Linux
: HI major with 60 votes (vote)
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-04 11:23 UTC by Heiner Markert
Modified: 2017-01-07 21:28 UTC (History)
8 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Example icalendar file showing the bug (526 bytes, text/calendar)
2011-12-17 19:59 UTC, Heiner Markert
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Heiner Markert 2011-11-04 11:23:06 UTC
Version:           4.7 (using KDE 4.7.2) 
OS:                Linux

I am using a kolab server with many events stored in the users calendars. I recently upgraded to KDE 4.7.2 (kubuntu-release).
Since the upgrade, recurring events that have been created using older versions of korganizer, or that have been created by e.g. syncml or the horde frontend to kolab, get displayed one hour off in either summer or winter time. Events starting in summer time display correctly during summer time, and are off by one hour in winter time, whereas events starting in the winter time get displayed correctly in winter time, and are off by one hour during summer time.
The display issues affect month view and partly day view: in day view, the graphical representation of the events is wrong, whereas in the title bar of the event, the correct time is displayed. When editing the event, correct times show in the edit fields.

Reproducible: Always

Steps to Reproduce:
Get an older version of korganizer not using akonadi, and connect it to a kolab server.
Create a recurring event.
Load the calendar in recent korganizer.
View the calendar, after a daylight saving time change (i.e., if the event starts in summer time, move to a date in winter time)

Actual Results:  
Event displays with one hour offset in tooltips of month view, "graphical representation" in day view. Title bar in day view shows correct times, editing the event also shows correct times.

Expected Results:  
Show correct times at all places!
Comment 1 Heiner Markert 2011-12-17 19:59:02 UTC
Created attachment 66838 [details]
Example icalendar file showing the bug

Add the following ics file as a calendar resource to Kontact.
Switch to day view, set locale to germany.
During summer time (e.g. in june), event is displayed at the right position in day view (at 7.30 am), during winter time (e.g. in december), the event is displayed at 6.30 am, which is wrong.
Comment 2 Sergio Martins 2011-12-17 20:29:04 UTC
DTSTART:20101001T053000Z
DTEND:20101001T064500Z


The attached file has an event starting at 05h30 (UTC), so, in the summer it should display at 07h30 ( because Germany is using GMT+2 ), and in the summer it should display at 06h30 ( because Germany is using GMT+1 ).

Throughout the year, the event is always displayed at UTC 05h30, because that's whats defined in DTSTART.

If you want the event to always appear at the same time in your local time, don't store events as UTC, use Europe/Berlin instead.
Comment 3 Heiner Markert 2011-12-17 21:16:16 UTC
Thank you for the quick response and the explanation. There is however another inconsistency:
When the event is double-clicked (i.e., modified), it displays as "7:30 Europe/Berlin" in the edit dialog - according to your explanation, I would expect "5.30 UTC" instead.

The event comes from Outlook 2003 originally - I will check whether or not it is possible to add the correct time zone information there and avoid UTC coding.

Best regards
Heiner
Comment 4 Bernhard E. Reiter 2012-01-03 13:13:55 UTC
Hi Sergio,
I believe the original report is that the behaviour of Kontact E35 and the new Kontact is behaving in consistently. This is a valid defect, in my view.
Kontact 4.7.x should behave consistently on Kolab recurring appointments
and use the old method which is to use the Kontact same daylight saving switching times. Of course with a new Kolab storage format, the situation will be even better, but the old one should be interpreted consistently.

The resulting ics file probably is just carrying already infected data.
Comment 5 Heiner Markert 2012-01-12 21:42:37 UTC
Hello,

as Bernhard posted above, the problem occurs in mixed client scenarios, where Kontact E35 and Kontact 4.7 clients access the same shared calendar through kolab. Recurring events created with Kontact E35 will not display correctly in Kontact 4.7.x in such a case.

Maybe an option that allows to select between the old and the new behavior could solve the problem?
Comment 6 itsef-admin 2012-03-01 14:37:49 UTC
(In reply to comment #5)
> as Bernhard posted above, the problem occurs in mixed client scenarios, where
> Kontact E35 and Kontact 4.7 clients access the same shared calendar through
> kolab. Recurring events created with Kontact E35 will not display correctly in
> Kontact 4.7.x in such a case.

As we just experienced, this problem not only occurs in set-ups where e35 and 4.7.x/4.8.x access the same resource concurrently, but also in migration scenarios. We just moved one of our users from Kontact e35 to Kontact 4.8.0 as part of a pilot and as a consequence, he is now complaining about a bunch of existing events that are shown (and "reminded of") at the wrong time (1h off). All of these are old events that were created in e35 before the migration. This may well be a show-stopper for us, preventing us from rolling out 4.8.x at all for the time being. :-(
Comment 7 Bernhard E. Reiter 2012-03-01 16:07:47 UTC
Heiner, Thomas, thanks again for your comments!
I have updated some properties accordingly like the severity
and I agree it is a clear defect that should be fixed.
The format itself has a version number, so the behaviour is well defined.
(And currently not as expected from Kontact 4.8 as you write.)
As I understood it Sergio did only commented looking at the ics file.

However Kontact Desktop from KDEPIM 4.8 and 4.9 is just coming out of a large
rearchitecture phase ranging several years. The process looking at defect
from the user point of view is just ongoing.
Comment 8 Christian Mollekopf 2012-05-07 21:23:45 UTC
Hey,
I couldn't really figure out yet where the defect lies, so I need some more info. I tried the supplied ics file in relatively recent build, which behaves generally correct, but the display of the times in the editor is not ideal:
- The editor shows the times in the local timezone instead of UTC (although the ics file contains utc time). The shown time is however correct and consistent with the shown timezone. Also does the editor not respect the summer/wintertime of the selected event, so if you're editing an event with wintertime during summertime, you open an event which is shown for 6.30 (wintertime) but get an editor showing 7.30 (summertime).
- The popup and the summary show the time correctly according to the active timezone.

Note that this was not tested with a kolab object but an ics file. If you still think there is a bug in korganizer or the kolabformat implementation, could you provide me with an event, preferably in kolab format (xml only or mime will do), and state what behavior you would expect and what you actually get from 4.8?
Of course you can send the file to me privately (mollekopf@kolabsys.com), I'll have a look at it then.
Comment 9 Bernhard E. Reiter 2012-05-08 08:14:42 UTC
Christian,
as pointed out in my Comment 4: "The resulting ics file probably is just carrying already infected data."

You need to test against Kontact E35 and the behaviour should be to display
the event as it is displayed in Kontact E35. This includes the event to stay
at the same "local" time during summer and wintertime. This even works in several timezone. Partly see http://wiki.kolab.org/KEP:2#Recurring_Events "make the implicit assumption that the time was specified in and should be calculated against the local time zone of the client itself."  and works in all timezone with the same delta hours and switchting dates. So all around Europe for example.
Comment 10 Christian Mollekopf 2012-05-08 10:37:40 UTC
The ics file contains perfectly valid data (generally speaking), dtstart is in 
UTC and is also interpreted accordingly, so at least in that respect there is 
no bug in recent Kontact versions. Of course it can still be inconsistent to 
e35.

So from my understanding the old behaviour is actually incorrect as it applied 
the dst changes first and then did the recurrence calculation from that 
starting point, while it should be the other way round. This means there is 
also no bug to fix in kontact.

What I think should be possible though is converting all events containing 
recurrences from UTC to floating time and adding the offset of the local 
timezone which was applicapable during dtstart of the event (and the other way 
around when saving). I have yet to check what e35 has been doing exactly, but 
something along the lines of this should reestablish the old behaviour.

Can you confirm that I understand the issue correctly?
I this case I would close this bug and open one against libkolab.
Comment 11 Bernhard E. Reiter 2012-05-08 10:56:00 UTC
Christian: 

First: Forget about the ics file. It is not about an ics file. It is not about ics behaviour. :)
Second: Yes, the Kontact E35 and old Webklient behaviour was suboptimal regarding some cases, but worked in other cases. E.g. in Europe. Behaviour was and is correct for thoses cases. This has been discussed at length in the KEP2 and related arguments documents. Again this is not the main point. :)

The main point is: Kontact E35 (and other Kolab Clients) for the Kolab Storage format v2
interpreted recurring events saved in a way. New clients must use the same interpretation (no matter how correct or incorrect it may be in the overal case).
So you have to check against Kontact E35 that the behaviour is the same.

As far as I know the behaviour is using the saved time as localtime in one timezone, but not floating. While the XML only has one UTC, this is meant as starting point of the recurrenace, so when there is a daylight saving in your primary client timezone, e.g. Europe/Berlin, you use the daylight saying so that the event stays fixed in both with and without dayslight saveing.
Comment 12 Christian Mollekopf 2012-05-08 13:50:59 UTC
This problem is about backwards compatibility in the kolabv2 implementation and will be resolved in libkolab. Kontact itself behaves correct.
The new report to track the issue is here: https://issues.kolab.org/show_bug.cgi?id=727

Thanks for the report.
Comment 13 Bernhard E. Reiter 2012-05-08 14:38:28 UTC
Christian, I believe I'd prefer to keep it open here, until it is resolved
in a regular Kontact distribution with a KDE software compilation.

Yes it is a defect in current Kontact as this has the Kolab Storage Format v2 interpreting
code which does an incorrect interpretation.
Comment 14 Christian Mollekopf 2012-05-08 17:24:37 UTC
Fair enough. Note that starting with 4.9 there is no kolabformat v2 interpreting code in Kontact anymore, as all that code was moved to libkolab. So this bugreport does not apply to Kontact 4.9.
Comment 15 itsef-admin 2012-09-06 15:56:49 UTC
We are now looking at Kubuntu 12.04/Kontact 4.8.4 - I take it that this problem has not been resolved so far?
Comment 16 Christian Mollekopf 2012-10-10 12:10:10 UTC
(In reply to comment #15)
> We are now looking at Kubuntu 12.04/Kontact 4.8.4 - I take it that this
> problem has not been resolved so far?

No it hasn't so far. I'm no sure if and when this is going to be fixed, as it is a compatibility problem with older kontact versions and behaves technically correct now. You are of course more than welcome to provide patches to http://git.kolab.org/libkolab/ to fix the issue with kolab v2 messages, or contact contact@kolabsys.com to free me some time to fix this for you.

Note that a fix in libkolab would only help for versions starting from Kontact 4.9 (the relevant code moved to a separate library).
Comment 17 itsef-admin 2012-11-27 15:08:25 UTC
I wonder - as this is a migration issue (and thus would - in our case - only happen once): Could a potential workaround be to come up with a script that modifies all relevant messages in the IMAP-spool upon migration? It's all text files, after all.
Comment 18 Christian Mollekopf 2012-11-27 16:12:19 UTC
(In reply to comment #17)
> I wonder - as this is a migration issue (and thus would - in our case - only
> happen once): Could a potential workaround be to come up with a script that
> modifies all relevant messages in the IMAP-spool upon migration? It's all
> text files, after all.

Unfortunately not, as it is a conceptual shortcoming of the Kolab v2 format.
The problem is that the format v2 stores all times in utc, so even if you adjusted the times in all files, it will always be wrong in either summer or wintertime. Therefore a workaround in the interpretation code would be required, which implicitly assumes all times as localtime (the timezone you're currently in), adjusts the utc-time accordingly, and calculates the recurrence from there.
This way the dst daylight savings can be applied during the recurrence calculation, resulting in an event which recurs for instance throughout the year at 13:00.

Of course that workaround would break UTC support (it would no longer be possible to store an event in utc), but I suppose you could live with that.

For a proper fix Kolab 3.0 is required which sports a new ical based format supporting timezones properly.
Comment 19 Bernhard E. Reiter 2012-11-28 10:02:32 UTC
I still hold that this is a defect in the Kontact Code (whereever it lives) that interprets the
Kolab v2 Format. The behaviour of the Kolab v2 format is consistent and documented
that it should be the same behaviour as Kontact E35 has it. So code that wants to
read Kolab v2 properly must implement this behaviour.

The v2 format has its shortcomings, so v3 will be better in this regard. Nevertheless
a correctly implemented v2 reader (and writer) would be fully interoperable with Kontact e35 here. 

I understand Christian in the way that the new libekolab does not have the Kolab
v2 storage format fully implemented so far and is imcompatible to the Kolab Format v2
in this point.
Comment 20 itsef-admin 2013-10-17 10:24:42 UTC
I just noticed that this bug has more implications that I thought - we are now looking at KDE 4.11.2 for migration from e35 and not only do all existing recurring events show up at the wrong time depending on DST, but the same goes for all *new* recurring events that I create with Kontact 4.11.2 as well! This is - as I just discovered - true for any "out of the box" event creation. Upon closer inspection, when I select "Time zones" in the event creation screen, the time zone is set to "UTC". If I change that manually to "Europe/Amsterdam", the event will be stored correctly. If I don't, it will be off by one hour in either summer or winter.
Comment 21 Denis Kurz 2016-09-24 20:34:13 UTC
This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.
Comment 22 Denis Kurz 2017-01-07 21:28:50 UTC
Just as announced in my last comment, I close this bug. If you encounter it again in a recent version (at least 5.0 aka 15.08), please open a new one unless it already exists. Thank you for all your input.