| Summary: | Adding a certain iCal resource makes Akonadi creates high CPU load | ||
|---|---|---|---|
| Product: | [Frameworks and Libraries] Akonadi | Reporter: | t.schmittlauch |
| Component: | ICal file resource | Assignee: | kdepim bugs <kdepim-bugs> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | auxsvr, carl, dvratil, winter |
| Priority: | NOR | ||
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Platform: | Gentoo Packages | ||
| OS: | Linux | ||
| Latest Commit: | https://invent.kde.org/pim/kdepim-runtime/-/commit/d136c91ace0f6e52089004221c0aabb696bdee63 | Version Fixed In: | 24.02.1 |
| Sentry Crash Report: | |||
|
Description
t.schmittlauch
2017-09-03 10:18:55 UTC
confirmed but I really don't think this is an Akonadi problem. either the libical low-level parsing lib or the KDE kcalcore abstraction library is at fault. > either the libical low-level parsing lib or the KDE kcalcore abstraction library is at fault.
Akonadi was my first guess as I could only observe which process had this issue.
Can you forward this bug report to the relevant project teams or shall I do that?
I found yet another ical file that causes high CPU load, this time mostly in korganizer itself. But when setting this resource to "offline" in Akonadi-Console, the problem vanishes in korganizer. So Akonadi (and maybe kcalcore) is definitely involved. This is the problematic calendar URL: https://unichor-dresden.de/index.php?option=com_chor&task=default.openCalendar&id=2 Also with https://calendar.google.com/calendar/ical/o4cqnocbu4ou1s5t556sec9hoo%40group.calendar.google.com/public/basic.ics. A similar bug a few years ago was that the akonadi resource would keep on refreshing for ever if it was marked as read only. Now it seems that after manually stopping refreshing, it resumes proper operation. The problem with the Google calendar is that every time it's fetched it's different, so the hashes don't match and we end up in an endless sync loop in the ical resource that fetches the ical, finds out that the hash differs from previous and triggers sync, which fetches the ical, finds out the hashes differ again and triggers another sync and so on. A possibly relevant merge request was started @ https://invent.kde.org/pim/kdepim-runtime/-/merge_requests/150 Git commit a4e290c26d98278f729073b909cc963e3331c0a8 by Daniel Vrátil. Committed on 28/02/2024 at 21:18. Pushed by dvratil into branch 'release/24.02'. Fix endless sync loop with some remote iCal sources The sync in SingleFileResource implementations (iCal, vCard and MBox resources) is implemented in a little strange way for remote files: 1) Akonadi (or someone) requests sync of the resource's collection 2) The resource starts a download of the new file 3) In the meantime, it syncs content of the current version of the file with Akonadi 4) When the file is downloaded, hash of the file is compared with previous hash and if they differ, goto 1 Some remote sources (e.g. Google Calendar) generate the iCal on-demand and as such the file always contains a different DTSTAMP. Thus, whenever the iCal resource completes the download, the hash will always differ compared to the previous one, and so it will again re-schedule sync over and over. While the data are actually available in Akonadi and to the user, it causes a high CPU load as the iCal is constantly being re-parsed, Akonadi database update, the Indexing agent is triggered over and over again. And the bigger the calendar, the worse it is. This fix changes the behavior by downloading the file, running the parser (iCal/vCard/mbox/...) and after that performing a regular sync, so basically: 1) Akonadi (or someone) requests sync of the resource's collection 2) The resource starts a download of the new file 3) When the file is downloaded, the resource syncs the content of the file with Akonadi. I have opted for using QPromise+QFuture combo to perform the async chaining of download->sync. It's the only solution that requires the last amount of code refactoring and reorganization in order to properly sequence the operations. This would be the perfect place for using coroutines, but I don't want to introduce coroutines as part of a regular bugfix. I might do an experimental port at some point later. FIXED-IN: 24.02.1 M +15 -6 resources/ical/shared/icalresourcebase.cpp M +42 -35 resources/mbox/mboxresource.cpp M +14 -6 resources/shared/singlefileresource/singlefileresource.h M +31 -28 resources/shared/singlefileresource/singlefileresourcebase.cpp M +8 -2 resources/shared/singlefileresource/singlefileresourcebase.h M +17 -13 resources/vcard/vcardresource.cpp https://invent.kde.org/pim/kdepim-runtime/-/commit/a4e290c26d98278f729073b909cc963e3331c0a8 Git commit d136c91ace0f6e52089004221c0aabb696bdee63 by Daniel Vrátil. Committed on 28/02/2024 at 21:42. Pushed by dvratil into branch 'master'. Fix endless sync loop with some remote iCal sources The sync in SingleFileResource implementations (iCal, vCard and MBox resources) is implemented in a little strange way for remote files: 1) Akonadi (or someone) requests sync of the resource's collection 2) The resource starts a download of the new file 3) In the meantime, it syncs content of the current version of the file with Akonadi 4) When the file is downloaded, hash of the file is compared with previous hash and if they differ, goto 1 Some remote sources (e.g. Google Calendar) generate the iCal on-demand and as such the file always contains a different DTSTAMP. Thus, whenever the iCal resource completes the download, the hash will always differ compared to the previous one, and so it will again re-schedule sync over and over. While the data are actually available in Akonadi and to the user, it causes a high CPU load as the iCal is constantly being re-parsed, Akonadi database update, the Indexing agent is triggered over and over again. And the bigger the calendar, the worse it is. This fix changes the behavior by downloading the file, running the parser (iCal/vCard/mbox/...) and after that performing a regular sync, so basically: 1) Akonadi (or someone) requests sync of the resource's collection 2) The resource starts a download of the new file 3) When the file is downloaded, the resource syncs the content of the file with Akonadi. I have opted for using QPromise+QFuture combo to perform the async chaining of download->sync. It's the only solution that requires the last amount of code refactoring and reorganization in order to properly sequence the operations. This would be the perfect place for using coroutines, but I don't want to introduce coroutines as part of a regular bugfix. I might do an experimental port at some point later. FIXED-IN: 24.02.1 (cherry picked from commit a4e290c26d98278f729073b909cc963e3331c0a8) M +15 -6 resources/ical/shared/icalresourcebase.cpp M +42 -35 resources/mbox/mboxresource.cpp M +14 -6 resources/shared/singlefileresource/singlefileresource.h M +31 -28 resources/shared/singlefileresource/singlefileresourcebase.cpp M +8 -2 resources/shared/singlefileresource/singlefileresourcebase.h M +17 -13 resources/vcard/vcardresource.cpp https://invent.kde.org/pim/kdepim-runtime/-/commit/d136c91ace0f6e52089004221c0aabb696bdee63 |