Bug 374780 - Huge memory usage just for the calendar with korgac and ical resource
Summary: Huge memory usage just for the calendar with korgac and ical resource
Status: RESOLVED WORKSFORME
Alias: None
Product: korganizer
Classification: Applications
Component: general (show other bugs)
Version: 5.2.3
Platform: Other Linux
: NOR major
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-01-08 20:27 UTC by Alexandre Bonneau
Modified: 2018-05-17 01:50 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
massif output for korganizer (789.75 KB, text/plain)
2018-01-05 11:10 UTC, ederag
Details
illegal type for property: VALUE=DATE-TIME (3.82 KB, text/plain)
2018-01-05 21:17 UTC, ederag
Details
start ics file (32 bytes, text/plain)
2018-01-24 02:26 UTC, Max
Details
UTC ics file (398 bytes, text/plain)
2018-01-24 02:26 UTC, Max
Details
non UTC ics file (5.94 KB, text/plain)
2018-01-24 02:27 UTC, Max
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandre Bonneau 2017-01-08 20:27:51 UTC
Reopened bug #329205 :
"At all time, I have 1 GiB of memory used by korganizer, korgac and akonadi_ical_resource processes, and only 72 MiB of it is shared.

The main memory hog seems to come from a duplication between the korganizer and korgac processes, respectively with 487 and 469 MiB used.

Here is my memory consumption using 4.11.2 :
Process | Memory | Shared memory | Virtual Memory
korganizer | 487.7 M | 31.0 M | 1.2 G
akonadi_ical_resource_1 | 5.0 M | 12.7 M | 308.4 M
akonadi_ical_resource_2 | 126.7 M | 13.5 M | 448.6 M
==> korgac | 469.0 M | 15.1 M | 1.0 G <==

The total memory usage for a single calendar is in my humble opinion far too high (more than 1,088 GiB) for a 6.7 MiB ics file.
This is specially true for netbook users.

More history about this bug can be found in bug 314900.

Reproducible: Always"

The problem is still present in v5.2.3, to a lesser extent, but korganizer/pim still uses way too much memory for not shown tasks/events, etc.

Updated grep commands :
`» grep BEGIN:VTODO calendrier.ics | wc -l
8553
» grep BEGIN:VJOURNAL calendrier.ics | wc -l
10
» grep BEGIN:VEVENT calendrier.ics | wc -l
5114
» grep BEGIN:JOURNAL calendrier.ics | wc -l 
0
» grep BEGIN:TODO calendrier.ics | wc -l    
0
» grep BEGIN:EVENT calendrier.ics | wc -l   
0`

Memory usage :
korganizer : 209 Mio
korgac     : 102 Mio
akonadi_ical_resource : 142 Mio
Total : 453 Mio of memory used...for a 6.6 Mio ics file.

Just to put that into perspective, that *68* time the size of the file on disk!!


If you ask yourself why the numbers of events/tasks did not go up since 2013, it's because the critical breaking bugs reported here are still present and still prevents the normal use of Korganizer.

I do think Korganizer can revive from its ashes and get back the pre-2013 version functionalities, hence my recent testing here..
Comment 1 ederag 2018-01-05 11:10:46 UTC
Created attachment 109689 [details]
massif output for korganizer

Same symptom here, although with an older version
korganizer --version
Qt: 4.8.6
KDE Development Platform: 4.14.25
KOrganizer: 4.14.10

korgac quickly reaches 2GB, then remains stable.

massif.out.16937 generated as follow:
akonadictl stop
kquitapp korgac
akonadictl start
valgrind --tool=massif korganizer --nofork
# the korgac process reaches about 2GB memory in few seconds
# could not quit cleanly the korganizer window (no interaction)
# Ctrl-C

# this is an ical, not a deprecated kcal issue
ps -ef  | grep akonadi | grep cal_
/usr/bin/akonadi_agent_launcher akonadi_ical_resource akonadi_ical_resource_4

# the .ics files are only few MB:
find .kde4 -name "*.ics" -exec du -sk {} \;
4       .kde4/share/apps/kalarm/calendar.ics
4       .kde4/share/apps/kalarm/displaying.ics
4       .kde4/share/apps/kalarm/expired.ics
2272    .kde4/share/apps/korganizer/std.ics
1124    .kde4/share/apps/ktimetracker/ktimetracker.ics

There might be something wrong in this database;
any advice for further investigation ?
Comment 2 ederag 2018-01-05 21:17:19 UTC
Created attachment 109701 [details]
illegal type for property: VALUE=DATE-TIME

Here is a clue: there were a lot of X-LIC-ERROR:

RDATE;VALUE=DATE-TIME:19230527T230000
X-LIC-ERROR;X-LIC-ERRORTYPE=PARAMETER-VALUE-PARSE-ERROR:Got a VALUE 
 parameter with an illegal type for property: VALUE=DATE-TIME
X-LIC-ERROR;X-LIC-ERRORTYPE=PARAMETER-VALUE-PARSE-ERROR:Got a VALUE 
 parameter with an illegal type for property: VALUE=DATE-TIME

kquitapp korganizer,
kquitapp korgac
akonadictl stop

removing them with kwrite
search/replace mode "Escape sequences" (on a single line, without quotes)
"\nX-LIC-ERROR;X-LIC-ERRORTYPE=PARAMETER-VALUE-PARSE-ERROR:
Got a VALUE \n parameter with an illegal type for property: VALUE=DATE-TIME" 
replace: empty
=>between 7000 and 8000 replacements...

akonadictl start
korganizer

korgac memory < 170 MB


As a check, brought back the previous std.ics. => 2GB.
New one => fine.

Unfortunately, each time akonadi is started or stopped, 
a new X-LIC-ERROR appear.
(one for each RDATE;VALUE=DATE-TIME line)

From this rate, it seems that the problem started less than 2 months ago.
System: openSUSE-Leap-42.2
The last update for kdepim was 5 months ago.
https://build.opensuse.org/project/show/openSUSE:Leap:42.2:Update
Maybe the last daylight change, about 2 months ago ?

the RDATE;VALUE=DATE-TIME:19230527T230000 format seems valid though
https://www.kanzaki.com/docs/ical/rdate.html
The years are weird (1923 ?), but removing those RDATE lines
make all December appointments shift by 1 hour for instance.
So they seem efficient somehow.

head_18105ca.ics, which contains only the VTIMEZONE,
is enough to reproduce the behavior.
Comment 3 Max 2018-01-23 15:13:43 UTC
I think my problem gained by same issue
I noticed my calendar on 1050 events got 6.5Mb when was added to korganizer 5.7
Earch event had timezone deferred
Comment 4 Max 2018-01-24 02:26:08 UTC
Created attachment 110082 [details]
start ics file
Comment 5 Max 2018-01-24 02:26:51 UTC
Created attachment 110083 [details]
UTC ics file
Comment 6 Max 2018-01-24 02:27:30 UTC
Created attachment 110084 [details]
non UTC ics file
Comment 7 Max 2018-01-24 02:29:30 UTC
Gentoo. Korganizer ver 5.7.1, KDE Frameworks 5.42.0, Qt 5.9.3

I think my problem gained by same issue
I noticed my calendar on 1050 events got 6.5Mb when was added to korganizer 5.7.1
The ics file was filled with "RDATE;VALUE=DATE-TIME:" entries.
Earch event had timezone differed from UTC.
I deleted all akonadi files and configs, cleaned up the ics-file. One was filled up again.
When I tried to add a calendar with UTC timezone events. The size of ics file wasn't changed.
Now I looking up for a workaround this issue
I added sample files with UTC and non UTC timezone
Comment 8 ederag 2018-03-11 15:40:54 UTC
The workaround described in comment #2 can be automated
(backup ~/.kde4/share/apps/korganizer/std.ics first !):

kquitapp korganizer
kquitapp korgac
akonadictl stop

# wait a bit for akonadi to really stop
sleep 2

# DISCLAIMER: This will modify the calendar IN-PLACE. BACKUP first !
# strip the .ics from wrong error messages
perl -i -p -0 -e 's/X-LIC-ERROR;X-LIC-ERRORTYPE=PARAMETER-VALUE-PARSE-ERROR:Got a VALUE \R parameter with an illegal type for property: VALUE=DATE-TIME\R//g' ~/.kde4/share/apps/korganizer/std.ics
Comment 9 Max 2018-04-21 07:00:24 UTC
With kde-apps/korganizer-18.04.0 kde-apps/akonadi-18.04.0 my calendars shrink from 6MB to 433KB and 1.3MB to 73 KB.
I think issue has been solved.
Thank developers so much

P.S. Kde-apps 18.04.0, kde-frameworks 5.45, gentoo
Comment 10 Christoph Feck 2018-05-17 01:50:38 UTC
Thanks for the update; let's close this. If you see this issue again with KDEPIM 5.8 (from KDE Applications 18.04), please add a comment.