Reassigning all KOrganizer bug reports and wishes to the newly created 
korganizer-devel mailing list.

there is now also RSCDS , which seems to be the easiest-to-install standalone calendar server (no complete groupware blabla, just calendaring, which is great for people having another email provider)

currently i use plain WebDAV with apache, but as my calendar grows, it starts to get more flaky (crashing, loosing data, errors "cannot save calendar" eg ...)

the RSCDS homepage is at:

http://rscds.sourceforge.net/

Google Calendar can now be a CalDAV server:
http://www.google.com/support/calendar/bin/answer.py?answer=99355

*** This bug has been confirmed by popular vote. ***

I support this bug report, it could be really nice to have support for calendarserver (http://trac.calendarserver.org/).
I tried to access to a calendarserver with a remote resource calendar on KOrganizer and sometimes it showed the calendar and other's not, and didn't upload correctly, so I supose that the remote resource calendar it's not the correct plugin for calendarserver, which is CalDAV based.

Yes, caldav support is important, please add it 

As kde rocks in design and functionnality, adding this to kontact would made it a killer app.

Also we need to stay at minimal the same level as evolution :-)
not under.

Contact me in private if funding could accelerate things .. 

+1

As Google Calendar is the only free and widely used online calendar service, it would be very important to add support for CalDAV in Kontact.

Critical feature, for me.

Open-Xchange plans to use CalDAV for their groupware, an right now we use DAViCal in our office to share calendars over the web. Please add CalDAV support, so I don't have to use Thunderbird! Kontact is a Mercedes Benz as far as groupware clients is concerned... Please make it a Rolls Royce... Thanks!

I'd really like to see this implemented. The google calendar is widely used and making it work with korganizer and kontact will get lot's of people in the boat.

Just cross-referencing with a wish for google-calendar sync.
Bug #125952
I'm voting for both of them.

CalDAV would enable use of Google Calender without any workarounds like GCALDaemon, so I vote for it too.

I vote for this also. Need to sync Kontact and Google Cal. 

This is being implemented in http://opensync.org/ and in its support for kde file based resources and Akonadi http://kdepim.kde.org/akonadi/

Eventually that modular design will allow all kind synchronization to different kind of storage.

@Juha: That are good news. Do you know anything about the development state?

http://www.opensync.org/wiki/status
http://www.opensync.org/roadmap
http://techbase.kde.org/Projects/PIM/Akonadi
http://techbase.kde.org/Schedules/KDE4/4.2_Feature_Plan#kdepim

I'm not very familar with Akonadi's progress, at least it's already included into Fedora 9 and Fedora 10. Opensync could use more people in testing, bug fixing and lot of its plugins don't have active maintainers atm.

I wouldn't hold my breath for OpenSync actually have a working, integrated implementation till 4.3.

I'm investigating implementing either this or syncing using Google's Data API.

I think bug 125952 is a dupe of this.

+1

I'm definitely voting for this.  With Google's CalDAV support, implementing this in Korganizer would greatly simplify my world.

This feature would be very helpful for Google Calendar users as GCalDaemon can not be the right way to connect to such an widely used system as Google's.

While the Akonadi approach might be reasonable for some, the interface between KOrganizer and Akonadi is still a bit cumbersome when compared to the direct KResource system.

The following project may provide a good starting point to develop a CalDAV connector:

http://sourceforge.net/projects/kresourcewcap/

I am willing to put in some time to modify this code to use libcaldav instead of WCAP.  However, I would appreciate some help from the others who are also interested in this functionality.  If I can find a couple of volunteers, we should be able to get it implemented.

Kumaran, I'm highly interested and willing to help you.  I'm running a DAViCal server, so perhaps that will help.

Since it is almost certain that KOrganizer will be switching to Akonadi at KDE 4.4, you might consider doing an Akonadi resource for CalDAV instead.

Actually it should be easier than a KResource plugin because it isn't necessary to fit the asynchronous network operations into a synchronous API contract.

A good starting point should be this tutorial: http://techbase.kde.org/Development/Tutorials/Akonadi/Resources

If you have any question consider posting to our development mailinglist kde-pim or asking on the #akonadi IRC channel (freenode network).

The Akonadi conversion has been on the horizon for quite some time now, which is one of the reasons why I did not embark on this project earlier.  While it may well be coming in KDE 4.4, that is still a long wait and almost certainly will be delayed by the inevitable issues that come up.  In the meantime, users of Kontact are still unable to access CalDAV calendars.

I did take a good look at Akonadi before posting my comment.  Unfortunately, the user experience in Kontact is not as smooth as when using the traditional KResources.  Perhaps this will change in the future, but realistically it will be another year before a solid Akonadi interface is available to users of downstream distributions.

I understand your frustration (and have my own set of frustrations with Akonadi). 

However, at this point Akonadi IS working in KDE 4.3, and even if you do create a kresource for CalDAV, it's likely to take almost as much time to trickle down in distributions as KDE 4.4

Thanks for your input.  Unfortunately, it's a matter of timeline.  I expect to stay on Fedora 11 (KDE 4.2) for some time.  The ideal situation would be for the CalDAV resource to be included in the KDE-PIM 4.2 source tree as an update.  However, a viable alternative would be to deploy it to users as a separate RPM.  Since I need CalDAV functionality now, writing the resource seems like the safest and most straightforward approach.

I agree.
The KResource plugin is important for everybody using KDE from 4.1 to 4.3, so it is definitely a good effort.

And it will take a while until even often updated distributions like Fedora or Ubuntu will ship KDE 4.4

Just wanted to be sure it is understood that people might also be interested in a solution for KDE 4.4 onwards.

Kevin,  Yes, i'm interested in KDE 4.3 and using Akonadi.  Kumaran and i have been in contact over email.  Not sure if we'll be working together on the same code, but at the very least we can trade notes.

Kumaran, when you develop the code, keep in mind that KDE 4.3 will be pushed to the Fedora repositories very soon. So you should better start off developing against 4.3, or at least not against 4.2only-features.

Thanks for the heads up.  I'm hoping to develop this as an independent plugin, packaged in a separate RPM.  Hopefully, it will work with little or no modification in KDE 4.3, provided that the kdepim-devel headers and libraries have not changed significantly.

Since KDE 4.3 has been pushed to Fedora 11, I will be developing the plugin against those development libraries.  The project is hosted on Google Code:

http://code.google.com/p/kcaldav/

I would appreciate if I could find a couple of volunteers to help me with development, since time constraints preclude me from completing the project by myself.  If you have a good C++ background with KDE or Qt experience, please contact me via email.

I've just trying the new caldav plugin for kde 4.3.1 with akonadi.
10 calendars registered ( hosted by a davical server ) are loaded very quickly.
(bunch of seconds compare to 2 minutes 15 for sunbird 1.0-pre )

I'm waiting the write access, (sorry I can't participate directly to developpement ) but can test many things.

It would be nice to be able to give the ressource a choosed name instead of default akonadi name ( exemple blue,red,green,black for the name of calendar which are actually name akonadi_ressource_2 .... ) 

I think it's important, when write would be able, to create a new event and choose in which calendar it would be registered ; the selection would be easier for human.
Don't know if it's already exist in upstream code.

(In reply to comment #34)
> I've just trying the new caldav plugin for kde 4.3.1 with akonadi.
May I ask you which caldav plugin you are using and where you got it from?

Sorry for this OT message, but I guess many people are interested in this.

I've just upgraded my basic opensuse 11.1 kde 4.2.4 package to 4.3.1 using the openbuild repository factory. ( I would recommend this only to some sort of poweruser as dependencies with other repository could be complex )

After that I've start the akonadi process see in systemsettings and add some ressources by the plugin caldav .

Hi,

Is there any design for this that you would like reviewed?  I would be happy to help review the planned communication flow between client and server for you - just point me at a wiki or something... :-)

Regards,
Andrew McMillan.

Hi All,

On a customer site, we are using davical (davical.org) with success ( server part ) from October 2007 until now.
I've dumped their database and load it ( after crypting real name in description and summary )
I've create a script which would permit me to update in the future the schedules proposed.

I've made a "public" test server, so we can test davical with a big bunch of data.
All events have really made by humans so they look more real. We have 8600 calendars items.

Actually the server run with apache2.0 + php5.2.9 + postgresql 8.1
Davical is the lastest stable 0.9.7.6 with awl 0.38
I've lots of bandwith, and cpu would be suffice.

Context :
On the customer site they have organized the work by team so we have 10 schedules ( 10 colors )
Every PC is connected using the SuperUser guy which is Assitant to every other schedules (and doesn't have it's own agenda)
There's on 60% of schedule event for all the days ( several events by day / by team )
Other are normal rendez-vous during the day.

The superuser guy was a necessary constraint due to the lack of sunbird able to connect with different user on the same host.
All users are in fact ressources.

So here are the uri

http://caldav-test.ioda.net/orange/home/
http://caldav-test.ioda.net/blue/home/
http://caldav-test.ioda.net/black/home/
http://caldav-test.ioda.net/magenta/home/
http://caldav-test.ioda.net/green/home/
http://caldav-test.ioda.net/red/home/
http://caldav-test.ioda.net/brown/home/
http://caldav-test.ioda.net/yellow/home/
http://caldav-test.ioda.net/cyan/home/
http://caldav-test.ioda.net/gray/home/


For using superguy assistant you need to log as superman and password ioda09

With this in place you can download (save with your caldav client) the collection.
As all manipulation is permitted you can also empty it, but I would prefer you create new event, and delete them afterwards.

Coming tomorrow : a way to see the debug part of php error log ( there are some ... )

I will also hack a bit the home page to re-explain all noticed here
+ put a link to download a pre-configured sunbird-1.0-pre profile

Hope all this effort will help us to make davical more usefull. more robust, And help caldav client to develop something better than what we have.
And boost somewhat the akonadi/ical plugin.


-- Bruno

(In reply to comment #38)

I'm just a silent observer here, but... wow... just wow... What a great effort! :)

Amazing.  This bug is now #3 on the most wanted features list.

I wrote up a the start of a design for kcaldav (or should that be 'kaldav' :-) here:

 http://code.google.com/p/kcaldav/wiki/Architecture

Now if someone with more capacity for KDE coding could get cracking we could have 3247 or so happy people... maybe even more :-)

Regards,
Andrew McMillan.

There is now a funded Elance job for this project:

http://www.elance.com/jobs/caldav_support_for_kontact/kde_linux/18684559

Interested developers should register with Elance to submit a proposal.

The current state of the Akonadi plugin is described here:
http://tokoe-kde.blogspot.com/2010/02/caldavcarddavgroupdav-support-for.html
So, current state seems to be Alpha/Beta, and with a bit of luck it will be shipped with KDE 4.5

A snapshot of the KCalDAV project can be downloaded here:

http://code.google.com/p/kcaldav/

This version contains read-only support for CalDAV calendars.  It would be much appreciated if users can provide feedback.  Please feel free to post any issues into the project issue tracker.

I too consider CalDAV essential, keep up the good work

Read-write support is complete and can be downloaded the project page:

http://code.google.com/p/kcaldav/

The project is now in the bugfix stage, so any feedback would be much appreciated.  The developer will be finishing the project in the next week.  Please post any bugs into the project's issue tracker so that he can resolve them.

The code has been developed on Fedora 11 and should also work on Fedora 12.  If volunteers are available, other distributions will be supported after final release.

For whoever is interested in testing I have created an Archlinux package: http://aur.archlinux.org/packages.php?ID=35055

I've tried it against Zimbra and couldn't get it working.

korganizer(9754)/kdepimlibs (kcal) KCal::ResourceCalendar::loadError: Error loading resource: "HTTP/1.1 302 Moved Temporarily
Server: nginx/0.5.30
Date: Fri, 26 Feb 2010 15:07:16 GMT
Content-Type: text/html
Content-Length: 161
Connection: keep-alive
Location: http://xxxxxxxxxxxxxx/dav/asn@redhat.com/Calendar

Let me know what I can do to provide more information. I have to leave now.

(In reply to comment #47)
The best way is if there is a public Zimbra server against which it doesn't work.  That information can be posted into the issue tracker so that the developer can reproduce it.

Would anyone be able to create RPMs for Fedora 11 and Fedora 12 with KDE 4.3?

libcaldav doesn't support https. But I'm adding the support right now.

I've made some patch to libcaldav to support https.
The tar.gz can be found in http://code.google.com/p/kcaldav/issues/detail?id=8

The test program from the library can now, show me events using an HTTPS URL, but I cannot see them in korganizer

For the interested readers:
An Akonadi-CalDav plugin is developed in:
http://websvn.kde.org/trunk/playground/pim/dav/

The difference to kcaldav: kcaldav can be used by Akonadi via a compatibility layer, but not native, since it works as a kressource and not as a Akonadi native plugin.

Can you please explain what does it mean for users? Which one should users prefer? Is using kcaldav as a kresource bad in any way?

There's a little trouble with utf8 envent. the display is wrong
Get Fermé!! Lundi de Pâques
in place of Fermé !! Lundi de Pâques
(Means Closed Eastern Monday)

Otherwise, it's very quick to display events.

Version 1.0 of KCalDAV has been released.

http://code.google.com/p/kcaldav/

If anyone would like to create packages for various distributions, I would be happy to post them on the project web page.

I have updated the Archlinux package. 

You can find it at: http://aur.archlinux.org/packages.php?ID=35055

I have been testing the resource and it seems to be working well.  At this point, I would appreciate some volunteers to help with the following:

1) Integration of the package into KDE
2) Creating RPM and DEB packages for distributions
3) Integration of packages into distributions

My time is quite limited, so I won't be able to handle all of this alone.  If we can work together, this bug can be finally put to rest to the benefit of all KDE users.

Just a note, last week caldav-test.ioda.net was inaccessible due to transfert on new server.

It's now re-open with more power to test. Go & abuse it !

Next week, it will be upgraded to the lastest davical release 0.9.9. 
it's actually run the 0.9.8 version.
and I will add a special cal which would contain 10000 events in one ics ( something call archives :-)

Is there anyone on this mailing list who can move forward the process for getting the connector integrated into KDE?  Similarly, can anybody help with Fedora and Ubuntu packages?

From past experience, the longer we wait, the more chance there is for code divergence.  By integrating it sooner rather than later, we can avoid debugging incompatibilities that will inevitably creep in with new KDE releases.

Any help is much appreciated.

Please add CalDAV native support to KOrganizer.

Yes, please add to KDE. It's working great, much better than the akonadi-caldav version, apart from TODO's. Even works great with Google Calendars via calDAV, again, better than the Akonadi Google Calendar agent which ONLY supports primary calendars at this stage.

The akonadi calDAV agent cannot handle google calendars, which it should, as a solution for secondary calendars.

There are currently two solutions for standalone calDAV servers around:
1) DAViCal (http://www.davical.org/)
2) SabreDAV (http://code.google.com/p/sabredav/)

Platform Version 4.4.92 (KDE 4.4.92 (KDE 4.5 RC2))
Kontact Version 4.4.5 with kcaldav-1.1.0

On 8/15/10, Ingo Ratsdorf <ingo@envirology.co.nz> wrote:
> https://bugs.kde.org/show_bug.cgi?id=133614
>
>
> Ingo Ratsdorf <ingo@envirology.co.nz> changed:
>
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |ingo@envirology.co.nz
>
>
>
>
> --- Comment #61 from Ingo Ratsdorf <ingo envirology co nz>  2010-08-16
> 00:13:57 ---
> Yes, please add to KDE. It's working great, much better than the
> akonadi-caldav
> version, apart from TODO's. Even works great with Google Calendars via
> calDAV,
> again, better than the Akonadi Google Calendar agent which ONLY supports
> primary calendars at this stage.
>
> The akonadi calDAV agent cannot handle google calendars, which it should, as
> a
> solution for secondary calendars.
>
> There are currently two solutions for standalone calDAV servers around:
> 1) DAViCal (http://www.davical.org/)
> 2) SabreDAV (http://code.google.com/p/sabredav/)
>
> Platform Version 4.4.92 (KDE 4.4.92 (KDE 4.5 RC2))
> Kontact Version 4.4.5 with kcaldav-1.1.0
>
> --
> Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are a voter for the bug.
>

I d'like to use CalDav with android
Please support.

Fabio

From following various google android newsgroups it would rather appear that this is an android problem. And google does not seem to be remotely interested in supporting it, similarly as per their google calendar which is a disaster in terms of calDAV compliance.

Ingo

> --- Comment #63 from  <bugs kde fv rimertis ch>  2010-11-20 14:29:41 ---
> I d'like to use CalDav with android
> Please support.
> 
> Fabio

Hi,

There is a Akonadi based CalDAV resource (DAV groupware resource) that is now usable and that implements access to calendars, todos and journal. The support for the first two has been tested and is stable and production-ready; for journal it should work equally well as it is using the same code base.

Actually the Google Calendars bugs have been addressed to the greatest extent possible from the resource, but there is still a known issue with secondary calendars : Google thinks that the organizer of the events must be the calendar, not the one you set… This cannot be addressed on our side, so please report to Google (if it hasn't been fixed already). Other than that it's working fine with Google.

As an extra bonus this resource also implements the CardDAV spec, as it's rather close to CalDAV, and that allows you to have only one resource to configure to get access to your events and contacts. It's also implementing GroupDAV for completness as there was a gap left where there used to be a KResource for this.

This resource is intended to ship with the next KDE PIM stable release, but you can already have a peek at it by using the latest beta with KDE SC 4.6RC2. Should you find any bug (eh, that's unlikely, right?) you have the possibility to create bug reports by using the 'DAV resource' component.

Cheers,
Grégory

Lets close this wish when kdepim is out then.

As for the kcaldav project, I don't see it getting into KDE: It's KResource based ( deprecated ) and uses KCal ( deprecated ).

I would encourage you not to close this wish so quickly.

I took some time to test the Akonadi CalDAV resource in 4.6 RC2 against DAViCal 0.9.9.4.  Unfortunately, my assessment is that Akonadi is not yet ready for a production desktop environment.

Here are the major points:

1) The performance is significantly worse than KCalDAV, especially on complex calendars.

2) I entered events which mysteriously disappeared, indicating that synchronization was not working properly.

3) Akonadi fails to launch on NFS-mounted home directories.

I realize that KResource will eventually be deprecated.  I would love to see Akonadi incorporate a stable CalDAV resource, removing the necessity for KCalDAV and my maintenance of it.  However, I'd like to note that Akonadi has had a history of breaking functionality with no viable workaround.  For example, the 4.3->4.4 address book change took many months to rectify in downstream distributions.
  
I chose to implement KCalDAV using KResource for this reason.  The KCalDAV plugin is primarily utilized by business users who cannot have the calendar broken for any reason.  Akonadi was (and still is in 4.6) too unstable to deploy into a corporate environment.

I will be posting DEB and RPM packages on the KCalDAV project website so that users can compare the functionality.  I verified that KCalDAV does work through the KResource compatibility system, so they can be easily tested side-by-side.

Hi,

> I took some time to test the Akonadi CalDAV resource in 4.6 RC2 against
> DAViCal 0.9.9.4. 

Excellent, thanks! For the rest of this mail I'll focus exclusively on 
problems that are resource-related, though thanks for taking time to give an 
overview.

> Here are the major points:
> 
> 1) The performance is significantly worse than KCalDAV, especially on
> complex calendars.

Are you talking about the resource performance here? If that's the case the 
only place where there is a "problem" is with the initial sync of calendars 
that contain a lot of entries, as all will be downloaded from the server. 
There is room here for some improvement, for example by fetching first the 
items that are around the current date, but there are some more features I'd 
like to see before starting this kind of optimizations. Besides this both 
resources should be on par.

> 2) I entered events which mysteriously disappeared, indicating that
> synchronization was not working properly.

Would you mind creating a bug report with the steps to reproduce please? I 
haven't seen this behavior in a long time (in fact this may have happened many 
months ago, or when using Google calendars, but these bugs have been addressed 
with the Google's limitation mentioned). I'll look into this as soon as I hear 
from you.

Cheers,
Grégory

Hi Grégory,

Thank you for the quick and detailed response.

> Are you talking about the resource performance here? If that's the case the 
> only place where there is a "problem" is with the initial sync of calendars 
> that contain a lot of entries, as all will be downloaded from the server. 
> There is room here for some improvement, for example by fetching first the 
> items that are around the current date, but there are some more features I'd 
> like to see before starting this kind of optimizations. Besides this both 
> resources should be on par.

There is probably a mix of resource and KOrganizer issues.  However, I do think that as a baseline improvement, the resource should not cache the entire calendar.  We came across this some time back during KCalDAV development.  Currently, we have DAViCal server with nearly a dozen calendars and thousands of simple and complex events.  Caching the entire calendar presents the following problems:

1) Very slow performance over remote VPN links.

2) Extra server overhead for NFS-mounted home directories.

3) Security concerns in the case of a lost or stolen laptop.

For these reasons, KCalDAV is set to cache events +/- 90 days from the current date.  This is a constant that can be tuned, but we have found that this value works pretty well for most situations.  Time-limited caching also has the desirable effect of preventing calendar lookups from becoming O(n) with respect to the size of the calendar.

From a user experience perspective, the primary test is how fast the calendar can be paged week-by-week.  Simply set the calendar to "Week" view and repeatedly click the right arrow to page to subsequent weeks.  For one calendar with over 500 events, my tests showed that KCalDAV renders the next week in about 250ms, while the Akonadi version takes over 1.5s.  This is a common use case for business users, since people often look for availability in the next few weeks when scheduling meetings.

> Would you mind creating a bug report with the steps to reproduce please? I 
> haven't seen this behavior in a long time (in fact this may have happened
> many months ago, or when using Google calendars, but these bugs have been
> addressed with the Google's limitation mentioned). I'll look into this as
> soon as I hear from you.

Unfortunately, I don't have a repeatable sequence of steps to recreate the problem.  I had a similar intermittent bug a while back in KCalDAV and was only able to resolve it by carefully reading the code and finding an error in the XML construction.  I also would not be able to provide access to the calendar which exhibited this problem, since it is internal to the company.  My best suggestion would be to create a large calendar (1000+ events, with recurrences and all-day events) and read/write simultaneously from multiple clients.  Perhaps you can read the code and spot something that might be causing the issue.

Best Regards,
Kumaran

Hi Kumaran,

> > Are you talking about the resource performance here?
> 
> There is probably a mix of resource and KOrganizer issues. 

There are even three actors in this play: Akonadi, the resource and KOrg. The 
resource is mostly just a simple worker that will download the calendar 
entries and that will discover new calendars.

> However, I do
> think that as a baseline improvement, the resource should not cache the
> entire calendar. 

This sounds counter-intuitive, as it may seem better, performance-wise, to 
have all the calendar on a local, faster, storage, rather than having to 
download new entries over the network.

> We came across this some time back during KCalDAV
> development. Currently, we have DAViCal server with nearly a dozen
> calendars and thousands of simple and complex events.  Caching the entire
> calendar presents the following problems:
> 
> 1) Very slow performance over remote VPN links.

I don't get this point: how is the VPN used in this scenario? I guess it's not 
to download new entries. With you next point in mind I imagine that you may be 
mounting users home with NFS over the VPN, but that's just a guess.

> 2) Extra server overhead for NFS-mounted home directories.

Yup, and Akonadi does not work well in this scenario. To be complete it's the 
MySQL server that does not deal well with NFS-mounted database files. I can 
see two points here that may be areas of investigation: using an external 
MySQL server (meh, not so good over VPN with high-latency / low-bandwidth 
links); or putting the MySQL database files on another location of the local 
drive. Spontaneously I think about /var/lib/akonadi-userdata/[user]/, but this 
has the drawback that moving from one machine to another will need an Akonadi 
reconfiguration each time… No ideal solution here.

> 3) Security concerns in the case of a lost or stolen laptop.

Though it's an interesting problem in itself I don't think that limiting the 
cached data size provides an interesting solution. If the theft is targeted 
then having 180 days (as you say later in your message) is ample enough to 
know the habits of the target, who is he in contact with, etc. One good answer 
to this problem is full disk encryption, though it's no silver bullet. But 
this is totally off-topic here :)

> From a user experience perspective, the primary test is how fast the
> calendar can be paged week-by-week.  Simply set the calendar to "Week"
> view and repeatedly click the right arrow to page to subsequent weeks. 
> For one calendar with over 500 events, my tests showed that KCalDAV
> renders the next week in about 250ms, while the Akonadi version takes over
> 1.5s.  This is a common use case for business users, since people often
> look for availability in the next few weeks when scheduling meetings.

I can see the problem here. 1.5s is clearly too long. I don't know for sure 
the internals of Akonadi, so what will follow is an educated guess. Akonadi 
stores items with many different mime types as opaque data: it does not 
process the content of events for example an is thus unable to optimize 
storage based on this content. For events this would be for example the start 
date / end date, attendees, and so on. Optimization would be to use MySQL 
indexes to speed up search, and having KOrg issue a special query to get 
relevant items for the currently selected timeframe, with a little fetch-ahead 
to be responsive and accomodate users navigating around in their calendars.

This is an interesting problem, and I have no sure idea of how to address it. 
Maybe KOrg can do the fetch-ahead (and fetch-behind) without special 
optimization on the MySQL side. Or maybe using Strigi may help searching for 
items.

[Items not created / synced]

> Unfortunately, I don't have a repeatable sequence of steps to recreate the
> problem.  I had a similar intermittent bug a while back in KCalDAV and was
> only able to resolve it by carefully reading the code and finding an error
> in the XML construction. 

Events creation is rather straightforward and does not involve XML: it's just 
using an HTTP PUT request (via KIO::storedPut) with the item payload as the 
request body and the appropriate If-None-Match HTTP header.

> I also would not be able to provide access to
> the calendar which exhibited this problem, since it is internal to the
> company. 

Eh, no problem, I understand :) This bug occured only in one calendar? Have 
you tested this with others?

> My best suggestion would be to create a large calendar (1000+
> events, with recurrences and all-day events) and read/write simultaneously
> from multiple clients. Perhaps you can read the code and spot something
> that might be causing the issue.

With this info I think there may have been a conflict in the items that were 
created. In this case it's the first created one that wins if the second does 
not properly set the If-None-Match header to match the first item ETag. A HTTP 
error (4xx) is sent back by the server and caught by the resource, but Akonadi 
does not display this to the user (at least that was still the case last time 
I checked), leading you to believe that the item was correctly created while 
in fact it was not. Looking at the web server logs may help here.

Cheers,
Grégory

(In reply to comment #70)

> I can see the problem here. 1.5s is clearly too long. I don't know for sure 
> the internals of Akonadi, 

Regarding the item above, I have a ticket open for one year:
https://bugs.kde.org/show_bug.cgi?id=215972
with no answer yet. Basically my 5 "heavily filled" local calendars are unusable via akonadi, since I need to wait 60-240 seconds before I can see the entries. In all this time kontact is frozen.

Hi Grégory,

> This sounds counter-intuitive, as it may seem better, performance-wise, to 
> have all the calendar on a local, faster, storage, rather than having to 
> download new entries over the network.

In the previous scheme, the cache is file-based, so searches would be dependent on the number of entries that were cached.  Since Akonadi uses a relational database, this may become less of an issue.  However, the purpose of a cache is to keep frequently used data locally, not create a full mirror of the thousands of entries on the server.  With this perspective, it seems crufty to cache thousands of events when they are unlikely to be used.

> I don't get this point: how is the VPN used in this scenario? I guess it's not 
> to download new entries. With you next point in mind I imagine that you may be 
> mounting users home with NFS over the VPN, but that's just a guess.

If somebody adds a new calendar while on VPN, this will become an issue.  Also, if a force refresh is required for whatever reason, anybody on a slow link will be waiting for a long time.  Referring to my point above, I feel that it's best to cache the minimum amount of data necessary to ensure good performance.

> Yup, and Akonadi does not work well in this scenario. To be complete it's the 
> MySQL server that does not deal well with NFS-mounted database files. I can 
> see two points here that may be areas of investigation: using an external 
> MySQL server (meh, not so good over VPN with high-latency / low-bandwidth 
> links); or putting the MySQL database files on another location of the local 
> drive. Spontaneously I think about /var/lib/akonadi-userdata/[user]/, but this 
> has the drawback that moving from one machine to another will need an Akonadi 
> reconfiguration each time… No ideal solution here.

We use NFS-mounted home directories so that a user can seamlessly login to any equivalent workstation.  This is just on the LAN, not over VPN.

I believe that the use of a full MySQL server for Akonadi was an unfortunate design decision at the very start.  Since Akonadi become a required component, it has caused nothing but problems in a wide variety of configurations.  We are currently forced to symlink the Akonadi directory to /var/tmp/kdecache-user/akonadi, losing the previous ability to login on any workstation.  We are working around this limitation by using only KResources so that the empty Akonadi database can simply be re-created when the user logs in to a different machine.

These issues are problematic enough that if it gets much worse, we may have to commit resources to enhance Evolution to the point where we can switch away from Kontact as our business messaging platform.

> Though it's an interesting problem in itself I don't think that limiting the 
> cached data size provides an interesting solution. If the theft is targeted 
> then having 180 days (as you say later in your message) is ample enough to 
> know the habits of the target, who is he in contact with, etc. One good answer 
> to this problem is full disk encryption, though it's no silver bullet. But 
> this is totally off-topic here :)

This is more of a business perspective.  While 180 days of data may be enough to surmise a user's habits, the point is that any breaches are limited by default to 180 days.  Management tends to feel better when they know that the potential security issues are at least limited to a well-defined time period.

> I can see the problem here. 1.5s is clearly too long. I don't know for sure 
> the internals of Akonadi, so what will follow is an educated guess. Akonadi 
> stores items with many different mime types as opaque data: it does not 
> process the content of events for example an is thus unable to optimize 
> storage based on this content. For events this would be for example the start 
> date / end date, attendees, and so on. Optimization would be to use MySQL 
> indexes to speed up search, and having KOrg issue a special query to get 
> relevant items for the currently selected timeframe, with a little fetch-ahead 
> to be responsive and accomodate users navigating around in their calendars.
>
> This is an interesting problem, and I have no sure idea of how to address it. 
> Maybe KOrg can do the fetch-ahead (and fetch-behind) without special 
> optimization on the MySQL side. Or maybe using Strigi may help searching for 
> items.

I feel that this points to a problem with the Akonadi architecture as a whole.  The KCalDAV resource is simple and fast.  Akonadi provides no additional functionality, yet it introduces performance issues.  KOrg should not have to work around Akonadi limitations.  The Akonadi system needs to be architected with a good user experience in mind.

> Eh, no problem, I understand :) This bug occured only in one calendar? Have 
> you tested this with others?

Sorry, I wasn't able to test it with others, since I wanted to avoid potential calendar corruption.

> With this info I think there may have been a conflict in the items that were 
> created. In this case it's the first created one that wins if the second does 
> not properly set the If-None-Match header to match the first item ETag. A HTTP 
> error (4xx) is sent back by the server and caught by the resource, but Akonadi 
> does not display this to the user (at least that was still the case last time 
> I checked), leading you to believe that the item was correctly created while 
> in fact it was not. Looking at the web server logs may help here.

While this explanation may be plausible, please keep in mind that the same use-cases work correctly with KCalDAV.  Given that Akonadi has not been tested thoroughly in a business environment, there may be a subtle unresolved issue in the resource.

Best Regards,
Kumaran

(In reply to comment #72)
> Hi Grégory,
> 
> > This sounds counter-intuitive, as it may seem better, performance-wise, to 
> > have all the calendar on a local, faster, storage, rather than having to 
> > download new entries over the network.
> 
> In the previous scheme, the cache is file-based, so searches would be dependent
> on the number of entries that were cached.  Since Akonadi uses a relational
> database, this may become less of an issue.  However, the purpose of a cache is
> to keep frequently used data locally, not create a full mirror of the thousands
> of entries on the server.  With this perspective, it seems crufty to cache
> thousands of events when they are unlikely to be used.

The cache has two use cases:
- having often used data accessible locally
- providing offline capability.

Using a cache timeout results in the first behavior, deactivating the timeout results in the second one.

Filling of the cache can be done in various ways, e.g. doing an initial full sync, or just getting a bunch of items and filling the rest later (with calendars a resource will have to get all items at least once to make sure it doesn't miss any which could apply to now or the near future due to recurrency rules).

> We use NFS-mounted home directories so that a user can seamlessly login to any
> equivalent workstation.  This is just on the LAN, not over VPN.
> 
> I believe that the use of a full MySQL server for Akonadi was an unfortunate
> design decision at the very start.

It is important to note that a common misconception about Akonadi's use of MySQL is that Akonadi requires a per-user instance of mysqld running.

While it is the default option for several reasons, there are quite a number of other ones, including running a central mysqld instance or on the application server or using file based SQLite, ...

> > I can see the problem here. 1.5s is clearly too long. I don't know for sure 
> > the internals of Akonadi, so what will follow is an educated guess. Akonadi 
> > stores items with many different mime types as opaque data: it does not 
> > process the content of events for example an is thus unable to optimize 
> > storage based on this content. For events this would be for example the start 
> > date / end date, attendees, and so on. Optimization would be to use MySQL 
> > indexes to speed up search, and having KOrg issue a special query to get 
> > relevant items for the currently selected timeframe, with a little fetch-ahead 
> > to be responsive and accomodate users navigating around in their calendars.
> >
> > This is an interesting problem, and I have no sure idea of how to address it. 
> > Maybe KOrg can do the fetch-ahead (and fetch-behind) without special 
> > optimization on the MySQL side. Or maybe using Strigi may help searching for 
> > items.
> 
> I feel that this points to a problem with the Akonadi architecture as a whole. 
> The KCalDAV resource is simple and fast.  Akonadi provides no additional
> functionality, yet it introduces performance issues.  KOrg should not have to
> work around Akonadi limitations.  The Akonadi system needs to be architected
> with a good user experience in mind.

Following some of the other comments on this report I am not sure which version of KOrganizer got used for getting the Akonadi calendar timing.
My guess is that it might be the same version used for timing the KResource based calendar, which obviously would make any numbers for the Akonadi calendar meaningless (as this would imply using the KResource wrapper plugin which has to load all calendars synchronously to work around this KOrganizer version's limitations).

> The cache has two use cases:
> - having often used data accessible locally
> - providing offline capability.

I agree with this requirement.  However, it is a good engineering practice to have a well-defined flushing policy for any cache.  For example, if there is a time window that will be cached, then objects outside that window should be discarded.  What I am uncomfortable with is the fact that the user has no control over the ever-growing cache files in the home directory.  A good example is the modern web browser cache which provides the user with sufficient amount of control over what is saved.

> It is important to note that a common misconception about Akonadi's use of
> MySQL is that Akonadi requires a per-user instance of mysqld running.
> 
> While it is the default option for several reasons, there are quite a number of
> other ones, including running a central mysqld instance or on the application
> server or using file based SQLite, ...

As I have posted in the past (i.e. KAddressBook 4.4), I feel that there has been a lack of user experience analysis on the part of the KDE PIM project.  Please take a step back and consider the statement above.  Akonadi requires the user to start an industrial-strength SQL database server just to store email, contacts, and calendar.  This design decision manifests itself as a tremendous amount of headache for the end user.  If you search on the Internet for Akonadi issues, you'll find a large number of users who can't even get the server started!

I am aware of the ability to use a central MySQL server, or even one on a different machine.  These are simply not reasonable options for the average user.  The best approach is to use a file-based (SQLite, BDB, ASCII) backend that can be easily tested to work on the majority of standard VFS mount points (local, NFS, AFS, Samba).  My experience in software architecture tells me that Akonadi is too complex and brittle for the tasks that it is intended to perform.

> Following some of the other comments on this report I am not sure which version
> of KOrganizer got used for getting the Akonadi calendar timing.
> My guess is that it might be the same version used for timing the KResource
> based calendar, which obviously would make any numbers for the Akonadi calendar
> meaningless (as this would imply using the KResource wrapper plugin which has
> to load all calendars synchronously to work around this KOrganizer version's
> limitations).

Akonadi calendar timing was conducted on KDE 4.6 RC2.  KCalDAV timing was conducted on Kontact 4.4.9.  My general point is that the reasons for the slow-down are somewhat irrelevant.  The user experience has been compromised to the point where the new version is not usable in a business environment.

Best Regards,
Kumaran

(In reply to comment #74)
> > The cache has two use cases:
> > - having often used data accessible locally
> > - providing offline capability.
> 
> I agree with this requirement.  However, it is a good engineering practice to
> have a well-defined flushing policy for any cache.

Agreed. Akonadi's cache flush policy is based on a timeout in minute granularity.

> For example, if there is a
> time window that will be cached, then objects outside that window should be
> discarded.

Right, a good policy for data that has some meaningful timestamp attached, e.g. mails, calendar entries. Doesn't make much sense for contacts though.
Easy for Email but tricky for calendar (due to requiring interpertation of recurrency rules).

> What I am uncomfortable with is the fact that the user has no
> control over the ever-growing cache files in the home directory.  A good
> example is the modern web browser cache which provides the user with sufficient
> amount of control over what is saved.

Good point. While clearing a resource's cache is already available programatically, this should be accessible in the UI as well.

> > It is important to note that a common misconception about Akonadi's use of
> > MySQL is that Akonadi requires a per-user instance of mysqld running.
> > 
> > While it is the default option for several reasons, there are quite a number of
> > other ones, including running a central mysqld instance or on the application
> > server or using file based SQLite, ...
> 
> As I have posted in the past (i.e. KAddressBook 4.4), I feel that there has
> been a lack of user experience analysis on the part of the KDE PIM project. 
> Please take a step back and consider the statement above.  Akonadi requires the
> user to start an industrial-strength SQL database server just to store email,
> contacts, and calendar.

See, that's why I used "common misconception". It is so common that it can even appear in the very context of a clarification that it isn't true.

But that is probably my fault, I tend to get clarifications too long. Let me try a shorter version:

Akonadi does not require the user to start an industrial-strength SQL database server.

> The best approach is to use a file-based (SQLite, BDB, ASCII) backend
> that can be easily tested to work on the majority of standard VFS mount points
> (local, NFS, AFS, Samba).

Which is why such options are supported by design. Nice, huh?

> Akonadi calendar timing was conducted on KDE 4.6 RC2.  KCalDAV timing was
> conducted on Kontact 4.4.9.

Ah, good.

>> The best approach is to use a file-based (SQLite, BDB, ASCII) backend
>> that can be easily tested to work on the majority of standard VFS mount points
>> (local, NFS, AFS, Samba).
>
> Which is why such options are supported by design. Nice, huh?

I did see this before posting and briefly tried the SQLITE backend, but Akonadi crashed and didn't start.  I will investigate further and post more information once I do a more controlled experiment.

I think the correct driver is called SQLITE3

Gentoo and Akonadi-on-WinCE defaults to this

Well, well, back to the original post topic:

There is support for caldav and groupdav. There is a groupdav Akonadi Resource in the KDE pim runtime repository https://projects.kde.org/projects/kde/kdepim-runtime/repository/revisions/master/show/resources/dav

I have been testing it for quite some time and it is awesome.

Just two issues with it:

It needs Kontact and KOrganizer to recognise deeper Akonadi folder structures:
BUG# https://bugs.kde.org/show_bug.cgi?id=232725

It needs a bugfix in Akonadi itself in order to changes to existing groupdav entries and I was told that this would happen in 4.6.3, which was released. SO: IS IT FIXED.

If the answer to bother items above is YES, DONE,

THEN

We have it.

Hi All,

With the release today of KDEPIM 4.6 I'm happy to give the bugreport for the second most wanted feature the mercy bullet. Hopefully you will no longer hear of this one :)

You are now able to sync with CalDav calendars by selecting the "DAV groupware resource".

From now on you can report new feature requests or bugs against Akonadi, component Dav groupware resource. Development is still ongoing and new features will be added.

Thanks to all the persons who already reported bugs against the new resource. Without your precious help and patience it wouldn't have been possible to reach the actual point and the resource would have been half-baked at best. Your help is most appreciated and you deserve a heartful "Thank you, you're awesome".

Cheers,
Grégory

As for Kubuntu 11.10 Oneiric alpha, I can confirm that the akonadi-groupdav-resource is working once you managed to download it's sources and compile.
For the first time ever, if works without crashes due to Kubuntu providing all the correct libraries.
Still many things bumpy and broken generally in Kubuntu (ALPHA!!), but that's finally working.

Want to install in Oneiric?
-------------------------------------------
Download from https://projects.kde.org/projects/kde/kdepim-runtime/repository
Download all of kdepim-runtime using git

Add at beginning of CMakeList.txt:

# Kubuntu still missing some desktop semantics libraries for successful compiling, broken dependencies
SET(KDEPIM_NO_NEPOMUK True)
#We want to install in correct place, not in /usr/local/bin, that's default
SET(CMAKE_INSTALL_PREFIX "/usr")

Change the following in CMakeLists.txt
#The runtime version they are asking for (4.7.49) is not yet available in Kubuntu, downgrade to 4.7, still works for the resource only
set( KDEPIM_RUNTIME_VERSION "4.7 ${KDEPIM_RUNTIME_DEV_VERSION}" )

Save and run cmake .
Pray!

If successful, run make akonadi_davgroupware_resource
Pray!

If again successful, change to ./resource/dav

Install using "sudo make install"

Add resource and enter information. At least in my installations, there is no final "Continue" or "Ok" or "Finish" button, only "back" and "cancel", so skip the wizard straight away and enter by hand. ;-)

----------------------------------------

Horray!!!

A big thanks for Grégory Oestreicher for getting this resource done.

Cheers,
Ingo

"DAV groupware resource" works perfectly with Google Calendar but it is tricky to set-up since Google, nor an "Others" option, is present on the set-up window. 

To configure it I had to look at
http://jpwhiting.blogspot.com/2011/04/kontact-with-google-calendar.html

Best regards,
Octavian

P.S. Before using "DAV groupware resource" I was using akonadi-googledata-1.2.0 with libgcal-0.9.6 which did not work properly as shown in http://code.google.com/p/libgcal/issues/detail?id=74