Bug 192187

Summary: The conduit Calendar Conduit could not be executed
Product: [Unmaintained] kpilot Reporter: Joe Christy <joe>
Component: Calendar ConduitAssignee: groot
Status: RESOLVED UNMAINTAINED    
Severity: normal CC: broeksema, joe, vR
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Fedora RPMs   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: log produced by command kpilotDaemon --debug=9 > kpilot.debug.7 2>&1
part 1 of log file produced by step 1 in comment #7
part 2 of log file produced by step 1 in comment #7
part 3 of log file produced by step 1 in comment #7
akonadi self-test report from step 2 of comment #7
akonadi self-test report from step 4 of comment #7
akonadi self-test report from step 6 of comment #7
KPilot UI log of Handheld to PC in step 8 of comment #7
kcrash log from PilotDaemon at step 9 of comment #7
log produced by "kpilotDaemon --debug=9" at step 11 of comment #7
kcrash log from PilotDaemon at step 12 of comment #7
log produced by "kpilotDaemon --debug=9" at step 13 of comment #7
KPilot UI log of HotSync in step 14 of comment #7
akonadi self-test report from step 15 of comment #7
akonadi self-test report from step 16 of comment #7
log produced by "kpilotDaemon --debug=9" at step 18 of comment #7
KPilot UI log of HotSync in step 19 of comment #7
~joe/.local/share/akonadi/db_data/mysql.err from experiment in comment #7
~joe/.local/share/akonadi/db_data/mysql.slow from comment #7
excerpt from log produced by kpilotDaemon --debug=9 2> kpilot.debug.2

Description Joe Christy 2009-05-10 01:25:29 UTC
Version:            (using KDE 4.2.2)
Compiler:          gcc-4.3.2 
OS:                Linux
Installed from:    Fedora RPMs

When I try to sync my Treo 650 w/ my Fedora 10 laptop neither the Calendar nor ToDo conduits will run. The Addressbook conduit works fine.

I have Akonadi setup, and, as far as I can see from the Akonadi Configuration Control Center self-test, it is functioning properly. I initialized the Akonadi resources from the .vcs and .ics files created by the kpilot from my KDE-3.5 fedora 8 setup.

Kpilot is configured to resolve conflicts by overwriting the PC with the Handheld data. I have purged all events & todos older than a week on the Treo, and deleted all completed ToDos and archived all calendar entries older than a week in Kontact.

Launching kpilot via:

kpilotDaemon --debug=9 > kpilot.debug.7 2>&1

on the commandline, in the UI, I see:

16:14:56 [Conduit kpilot-conduit-calendar] 
16:15:01 Invalid record mapping. Doing first sync.
16:15:01 Could not open or create PC data store.
16:15:01 The conduit Calendar Conduit could not be executed.
16:15:01 [Conduit kpilot-conduit-todo] 
16:15:02 Invalid record mapping. Doing first sync.
16:15:02 Could not open or create PC data store.
16:15:02 The conduit To-do Conduit could not be executed.

Unfortunately, this bug report wizard will not accept the file kpilot.debug.7, or even just the relevant excerpt [kind of puts the lie to "The more useful information you have, the better!" next to the input box]. I am eager to submit the debugging info, if someone could suggest how.

Curiously, despite all purging and archiving, while the Treo reports only 48 records in its Calendar/ToDo db, kpilot reports tring to sync 674 records.

Extremely unfortunately, there is no Akonadi documentation, and I was unable to google up either the location of the MySQL db backing Akoandi, or the schema of the database, from the Internet. Otherwise, I'd try flushing that db and starting fresh.
Comment 1 Joe Christy 2009-05-10 01:28:03 UTC
Created attachment 33503 [details]
log produced by command kpilotDaemon --debug=9 > kpilot.debug.7 2>&1
Comment 2 Jason 'vanRijn' Kasper 2009-05-10 05:31:18 UTC
Hi Joe. Hm. Did you configure KPilot to use the new akonadi data resources for these conduits?
Comment 3 Joe Christy 2009-05-10 19:03:15 UTC
As I said, as far as I can tell [cf. bug 192188], I've configured KPilot to use the akonadi data resources properly. The Contacts conduit visibly works, the akonadi self-test succeeds, and what akonadi logs I can find show no errors. The only visible funkiness is functional, in the UI, and recorded in the debug log from kpilotDaemon.

There must be some way to extract more information from akonadi, but I haven't been able to discover how. Sigh.

Does anyone know how to verify that KPilot is properly configured to use akonadi, or, even better, willing to document how to configure KPilot to use akonadi in the first place. I'd be glad to write documentation, but I literally don't know what I'd be documenting.

Jason - any help would be warmly appreciated. Thanks.
Comment 4 Jason 'vanRijn' Kasper 2009-05-11 00:39:38 UTC
Hi Joe, KPilot should be configured to use akonadi if you open KPilot's configuration settings for the conduits and select your resource from the dropdowns available. If you've done that much, then it should be configured properly. I see this in the log:

           > loadAllRecords 
             > event 
               > event 
             > addMessage 
             > addMessage 
             > addMessage 
             > changeIcon 
             > addMessage 
              Could not load records, is akonadi running? 
         > isOpen 
!  
!    Error: Could not fetch collection with id:  2 


So, this means that KPilot is not able to load the records from Akonadi for the resource that it's been configured to use.
Comment 5 Joe Christy 2009-05-11 01:58:08 UTC
Hi Jason -
in System Settings > Advanced > Akonadi Configuration > Akonadi Resources Configuration > Add > ICal Calendar File > Select Calendar panel I selected the vCalendar file ~/Library/Calendar/JPC.ics as "the file which content should be represented by this resource". This is the file that the KDE 3.5 KPilot was using as its store.

In KPilot > Settings > Configure KPilot ... > Conduits > Calendar panel (and likewise with To-do), I selected the only resource available in the popup, JPC.ics.

This is strictly parallel to what I did with the the Akonadi Vcard file, JPC-contacts.vcf from KDE-3.5 KPilot, and the Contacts conduit, which works fine.

Should I have done something different?

Is there a good way to start fresh and just copy over the data from my Treo into Akonadi?

On a different machine, I have tried selecting non-existant .ics and .vcf files, since the Akonadi Confguration UI seems to indicate in the selector that "If the file does not exist yet, it will be created", and selecting those from the KPilot Conduit config drop-downs, but when I synced, the files were *not* created and all the data on my Treo was *erased*, forcing me to restore from backups using pilot-xfer.
Comment 6 Jason 'vanRijn' Kasper 2009-05-11 17:35:47 UTC
Hi Joe, it sounds like you've done exactly the right things in setting this up. I'm at a loss for why it is not working. =:( I know that's not much help, and I apologize. I'll leave this bug open in hopes that in the near future, I might be able to steal some time away from life to look into what could be going wrong here. =:(
Comment 7 Joe Christy 2009-05-13 17:55:09 UTC
Thanks for your help, Jason.
Digging around a little, I found where akonadi keeps its db, so I removed it and tried the following experiment to test what happens when starting with a fresh akonadi installation in ~joe and my Treo reduced down to only the most recent and future events & to-dos.

More precisely:
0.  While logged in as root, delete all akoanadi traces in ~joe:
	.config/akonadi .kde/share/config/akonadi* .local/share/akonadi
    Log in as joe, forcing rebuild of the above files.
1.  launch kpilotDaemon --debug=9 2> kpilot.debug.1
2.  run akonadi self-test
3.  configure resources to use :
	~/Library/{Calendar/JPC-calendar.ics,/Contacts/JPC-contacts.vcf}
4.  run akonadi self-test. 
5.  restart akonadi
6.  run akonadi self-test.
7.  confirm kpilot to using akonadi resources
8.  use KPilot UI to copy HH to PC
9.  Unplug HH and KpilotDaemon crashes
10. Open Kontact and see Calendar and ToDos blank, Contacts OK.
11. restart kpilotDaemon --debug=9 2> kpilot.debug.11
12. try to HotSync & KPilot crashes
13. restart kpilotDaemon --debug=9 2> kpilot.debug.13
14. HotSync
15. kill pilot daemon to do post-mortem
15. Launch Kontact - no Contacts or Calendar
16. Restart akonadi-server
17. Add new appointment and edit an existing contact
18. restart kpilotDaemon --debug=9 2> kpilot.debug.18
19. HotSync
20. kill pilot daemon to do post-mortem

The following attachments are various log files produced, with a number at the end indicating the relevant step above. I've also attached the mysql.err and mysql.old from ~joe/.local/share/akonadi/db_data produced during the experiment.
Comment 8 Joe Christy 2009-05-13 18:23:02 UTC
Created attachment 33616 [details]
part 1 of log file produced by step 1 in comment #7
Comment 9 Joe Christy 2009-05-13 18:45:18 UTC
Created attachment 33618 [details]
part 2 of log file produced by step 1 in comment #7
Comment 10 Joe Christy 2009-05-13 18:49:10 UTC
Created attachment 33619 [details]
part 3 of log file produced by step 1 in comment #7

cat attachments 33616, 33618, and this one together to get the complete file, in order.
Comment 11 Joe Christy 2009-05-13 18:50:35 UTC
Created attachment 33620 [details]
akonadi self-test report from step 2 of comment #7
Comment 12 Joe Christy 2009-05-13 18:51:29 UTC
Created attachment 33621 [details]
akonadi self-test report from step 4 of comment #7
Comment 13 Joe Christy 2009-05-13 18:53:07 UTC
Created attachment 33623 [details]
akonadi self-test report from step 6 of comment #7
Comment 14 Joe Christy 2009-05-13 18:55:20 UTC
Created attachment 33624 [details]
KPilot UI log of Handheld to PC in step 8 of comment #7
Comment 15 Joe Christy 2009-05-13 18:56:20 UTC
Created attachment 33625 [details]
kcrash log from PilotDaemon at step 9 of comment #7
Comment 16 Joe Christy 2009-05-13 18:58:47 UTC
Created attachment 33626 [details]
log produced by "kpilotDaemon --debug=9" at step 11 of comment #7

log produced by command:
kpilotDaemon --debug=9 > kpilot.debug.11 2>&1
Comment 17 Joe Christy 2009-05-13 18:59:55 UTC
Created attachment 33627 [details]
kcrash log from PilotDaemon at step 12 of comment #7
Comment 18 Joe Christy 2009-05-13 19:01:03 UTC
Created attachment 33628 [details]
log produced by "kpilotDaemon --debug=9" at step 13 of comment #7
Comment 19 Joe Christy 2009-05-13 19:03:03 UTC
Created attachment 33629 [details]
KPilot UI log of HotSync in step 14 of comment #7
Comment 20 Joe Christy 2009-05-13 19:04:21 UTC
Created attachment 33630 [details]
akonadi self-test report from step 15 of comment #7
Comment 21 Joe Christy 2009-05-13 19:05:22 UTC
Created attachment 33631 [details]
akonadi self-test report from step 16 of comment #7
Comment 22 Joe Christy 2009-05-13 19:07:12 UTC
Created attachment 33632 [details]
log produced by "kpilotDaemon --debug=9" at step 18 of comment #7
Comment 23 Joe Christy 2009-05-13 19:08:23 UTC
Created attachment 33633 [details]
KPilot UI log of HotSync in step 19 of comment #7
Comment 24 Joe Christy 2009-05-13 19:10:13 UTC
Created attachment 33634 [details]
~joe/.local/share/akonadi/db_data/mysql.err from experiment in comment #7
Comment 25 Joe Christy 2009-05-13 19:11:23 UTC
Created attachment 33635 [details]
~joe/.local/share/akonadi/db_data/mysql.slow from comment #7
Comment 26 Joe Christy 2009-05-13 19:14:13 UTC
Two things that I forgot to mention:
0) the machine has a dual core x86_64 CPU & (Fedora 10) OS
1) After step 19 in comment #7, my Contacts re-appeared in Kontact, no joy on the Calendar & ToDos
Comment 27 Joe Christy 2009-05-27 01:12:04 UTC
FWIW, problem persists after upgrade to KDE-4.2.3/KPilot-5.2.3

...
16:02:27 [Conduit kpilot-conduit-calendar] 
16:02:28 HH data proxy: Start: 62. End: 61. 1 deleted record(s). 
16:02:28 PC data proxy: Start: 0. End: 24. 24 new record(s). 
16:02:28 Data mapping invalid after sync. Sync failed.
16:02:28 The conduit Calendar Conduit could not be executed.
16:02:28 [Conduit kpilot-conduit-todo] 
16:02:29 HH data proxy: Start: 17. End: 4. 13 deleted record(s). 
16:02:29 PC data proxy: Start: 0. End: 3. 3 new record(s). 
16:02:29 Data mapping invalid after sync. Sync failed.
16:02:29 The conduit To-do Conduit could not be executed.
...
Comment 28 Thomas 2009-05-28 19:38:21 UTC
I have recently experienced this bug too in Fedora 11 preview.  The synchronization was working only until recently.  I have tried just about everything that I can think of short of dropping the mysql DB, which I have done previously and I would really like to avoid since I worked real hard to get my contacts formatted properly so that they can be actually usable in my Garmin iQue.

Syncing contacts works with no issue.
Comment 29 Bertjan Broeksema 2009-05-30 13:18:18 UTC
Hi Joe,

Looking through your logs:

> IDMapping 
  > loadMapping 
    File does not exist, empty map.

Hmm... this should trigger a firstSync()

> addMessage 
!  
!    Error: Could not fetch collection with id:  2

This definitely looks like that the collection does not exist anymore. When you create remove an Akonadi agent and add a new one (even if it is for the same file), it will get a new id and you'll have to reconfigure KPilot (i.e. explicitly select the resource in the conduit config dialog. 

NOTE TO SELF: I need to fix this, so that it is more clear that the currently selected resource is not the same as the configured one in case of deleted/readded Akonadi resources.

> counter 
  HH data proxy:  "Start: 50. End: 50. No changes made. "
> counter 
  PC data proxy:  "Start: 0. End: 11. 11 new record(s). "

This does not look rigth =:/. Could you remove the mapping files (at least the one for Calendar) in:

~/.kde4/share/apps/kpilot/conduits/{USERNAME}/mapping

to be sure that a first sync is triggered.

Further more I saw that the 647 records don't come from your palm but from the Akonadi resource. Could you test with an Akanadi resource that doesn't have any records in it?

e.g.
1) # touch path/to/empty-calendar-file.ics
2) Configure a new ical resource pointing for path/to/empty-calendar-file.ics
3) Configure KPilot ical conduit to use the resource configured in #2
4) Make sure that no mapping file exists
5) Try to sync.
Comment 30 Thomas 2009-05-31 19:13:11 UTC
> > addMessage 
> !  
> !    Error: Could not fetch collection with id:  2
> 
> This definitely looks like that the collection does not exist anymore. When you
> create remove an Akonadi agent and add a new one (even if it is for the same
> file), it will get a new id and you'll have to reconfigure KPilot (i.e.
> explicitly select the resource in the conduit config dialog. 

This looks to have been my issue.  My missing collection had ID 3.  After several attempts to get a new collection recognized and sync'd, I finally succeeded; the unfortunate but liveable consequence was that my calendar events were duplicated.  As I reflect back, I was monkeying around with a second collection of my Kontact calendar entries for whatever reason and I suppose when I was finished, I removed the wrong Akanadi collection.

Thanks; your comments were very helpful.

Thomas
Comment 31 Joe Christy 2009-06-01 01:15:50 UTC
Well I did everything exactly as you said, but it didn't trigger a FirstSync, so I did it all again, and removed the Calendar and ToDo .pdb files from my conduits directory, which did trigger a FirstSync, but I still get:
15:57:29 [Conduit kpilot-conduit-calendar] 
15:57:30 Invalid record mapping. Doing first sync.
15:57:30 Could not open or create PC data store.
15:57:30 The conduit Calendar Conduit could not be executed.
15:57:30 [Conduit kpilot-conduit-todo] 
15:57:31 Invalid record mapping. Doing first sync.
15:57:31 Could not open or create PC data store.
15:57:31 The conduit To-do Conduit could not be executed.
 in the UI.

Looking at the log file from running the KpilotDaemon in debug -9, I'm still getting those messages about collection with id=2 no being found.

NB at no time did a new mapping file get generated for either Calendar or ToDo.

Which leaves the question: what are these akonadi collections? where are they supposed to be?
Comment 32 Joe Christy 2009-06-01 01:21:48 UTC
Created attachment 34155 [details]
excerpt from log produced by kpilotDaemon --debug=9 2> kpilot.debug.2

Could the problem in fact be the bit about

         > createDataStore 
            We don't support creation of akonadi datastores yet. Not doing anything. 
          Could not open or create pc data store.
Comment 33 Joe Christy 2009-06-01 01:23:04 UTC
NB I'm still eager to fix this problem, but will be on travel for the next week, so unable to respond until after June 6.
Comment 34 Bertjan Broeksema 2009-06-01 20:00:31 UTC
Chris,

The problem is this:

Could not load records, is akonadi running? 
> isOpen 
!  
!    Error: Could not fetch collection with id:  2 

You have configured an Akonadi Collection that doesn't exist.

1) Make sure you have an Akonadi collection for an calendar resource.
   (i.e. akonaditray -> configure -> add an ical (or other calendar agent) 
   agent/resource.
2) configure kpilot -> kpilot -s.
   a) Select the calendar conduit.
   b) Explicitly select the collection you just added in the combobox.
   c) Click apply.
3) Now start a sync.

The important thing is that you make sure that KPilot is configured to use the right resource. If the resource does not exists you get the message from the logs you've posted.
Comment 35 Thomas 2009-06-01 21:23:17 UTC
Well, when I finally corrected mine, I think I ended up with two calendar collections pointing to the same .ics file.  Then in the pilot config, I selected the newer collection, sync'd, deleted the old collection, sync'd, and started cleaning up the duplicates.

It was messy and no doubt I should have backed up the .ics records in a separate file, created a second, empty collection (or with one dummy entry), selected it in kpilot, sync'd where desktop overwrites the PDA, then copy and paste the backed up version of the original records into the new collection's .ics file, and sync'd where desktop overwrites hand held again.  I'm not going to try this myself since mine is working again and I don't want to risk having to clean it all up again.

Thomas
Comment 36 Bertjan Broeksema 2009-06-01 23:13:08 UTC
SVN commit 976515 by bbroeksema:

Give the user a hint in the configuration dialog when an Akonadi resource is
configured that doesn't exist any longer.

CCBUG: 192187


 M  +108 -54   akonadi-setup-widget.ui  
 M  +40 -10    akonadisetupwidget.cc  


WebSVN link: http://websvn.kde.org/?view=rev&revision=976515
Comment 37 Christophe Marin 2009-12-11 17:14:09 UTC
Unfortunately, KPilot is unmaintained due to lack of manpower and will not be part of KDE SC 4.4.

This bug reports will now be closed.