Bug 284087 - Memory leak in updating Magnatune database
Summary: Memory leak in updating Magnatune database
Alias: None
Product: amarok
Classification: Unclassified
Component: Services/Magnatune (show other bugs)
Version: 2.6.0
Platform: Ubuntu Packages Linux
: NOR major with 10 votes (vote)
Target Milestone: 2.7
Assignee: Amarok Developers
Depends on: 301349
  Show dependency treegraph
Reported: 2011-10-15 12:46 UTC by Marcus Harrison
Modified: 2012-08-18 12:34 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 2.7

Valgrind output (133.38 KB, application/octet-stream)
2011-10-15 19:42 UTC, Marcus Harrison
Valgrind output from Amarok 2.5-git of 21.2.2012 (220.57 KB, application/x-bzip)
2012-02-21 16:58 UTC, Myriam Schweingruber

Note You need to log in before you can comment on or make changes to this bug.
Description Marcus Harrison 2011-10-15 12:46:34 UTC
Version:           2.4.3 (using KDE 4.7.2) 
OS:                Linux

When updating the Magnatune database, either because it was done automatically or because one prompts the application to do it manually, the Amarok process continues to grab memory until it has finished updating the database, to such a degree that it climbed from ~30M to >200M in a few seconds. On my teeny tiny netbook, this has more than once caused Amarok to be closed by the kernel because my netbook ran out of memory. This has caused me to lose configuration settings and could potentially cause the loss of other data, if the magnatune database is updated multiple times in a single run of the application.

Reproducible: Always

Steps to Reproduce:
Update the magnatune database

Actual Results:  

Expected Results:  

OS: Linux (i686) release 3.0.0-12-generic
Compiler: gcc
Comment 1 Myriam Schweingruber 2011-10-15 17:01:16 UTC
We would need a valgrind report for that memory leak, please have a look here:

Since you didn't specify the technical details of your netbook it is rather hard to determine if Amarok is really suited for that platform, some netbooks are clearly under-equipped in terms of CPU power and/or memory.
Comment 2 Marcus Harrison 2011-10-15 19:42:07 UTC
Created attachment 64557 [details]
Valgrind output

I ran Amarok through Valgrind, navigated from my local collection to the Magnatune collection, clicked, "Reload Database" and then left it to run until the process ran out of memory and was killed (the process was using 80% of my 1G of memory at the time).
Comment 3 Myriam Schweingruber 2011-10-16 10:40:38 UTC
Could you please run Amarok without any active dynamic playlist, loaded playlist, running APG or 3rd-party script active? Please also do not run any other application to free as much memory as possible to get valgrind to actually finish its process.
We also need the system specifications as asked previously in comment #1
Comment 4 Myriam Schweingruber 2011-11-07 12:26:49 UTC
Closing for lack of feedback. Feel free to reopen if you can reproduce this
with the latest stable or git build.
Comment 5 Marcus Harrison 2012-01-26 22:18:03 UTC
I have a valgrind log, but it excedes the 1000Kb filesize limit for bugzilla, paste.kde.org and pastebin.com. Any suggestions where I could upload it?
Comment 6 Myriam Schweingruber 2012-01-27 07:10:46 UTC
Again, please provide the system specifications you were asked for. You can send the file to me by mail compressed with bzip2.
Also, is this still with Amarok 2.4.3? Then please upgrade to Amarok 2.5 first, as there were some significant changes in term of memory management.
Comment 7 Tamás Németh 2012-02-17 10:24:10 UTC
I have a dual-core, hyperthreading Intel Core i5 460M processor and 4GB of RAM, but Amarok is still able to consume my resources :-(

Amarok 2.4.3 consumes almost 300MB upon the first start after logging in to KDE. When I close and restart it, then its memory usage goes down to 30-40MB. Amarok 2.5.0 behaves similarly, but consumes even more memory! Upon its first start in the KDE session it consumes almost 400MB memory(!), but after a restart it goes down to approximately 50MB. Its (Amarok 2.5.0's) entire virtual memory map is bigger than 1.5GB!!!
Comment 8 Marcus Harrison 2012-02-17 14:09:30 UTC
Sorry, I forgot to provide information about my set-up. I thought I had sent it in the E-mail.

I'm using a laptop with an Intel Atom 64bit duel-core processor with 2GB of RAM. Previously I had been running a netbook with an Intel Atom 32bit duel-core processor with 1GB of RAM.
Comment 9 Myriam Schweingruber 2012-02-21 16:58:51 UTC
Created attachment 68979 [details]
Valgrind output from Amarok 2.5-git of 21.2.2012

Find attached my valgrind output with the options:

valgrind --tool=memcheck --leak-check=yes --track-origins=yes -v amarok -d  --nofork
Comment 10 Myriam Schweingruber 2012-02-21 16:59:22 UTC
Setting status correctly.
Comment 11 Emilio 2012-08-16 21:30:25 UTC
Still valid in 2.6.
In my case the memory consumption is increased from 80 mb to 400 mb.
Comment 12 Myriam Schweingruber 2012-08-16 23:24:58 UTC
Thank you for the feedback. Could you please check which phonon backend you are using? If you use the gstreamer backedn, please isntall the vlc backend, select it in systemsettings -> Multimedia -> Phonon -> Backend tab and restart KDE.

In my case the switch to the vlc backend reduced the memleak drastically.
Comment 13 Emilio 2012-08-17 11:34:24 UTC
I tested phonon-gstreamer-4.6.2 and phonon-vlc-0.5.0 in Mageia 2, with KDE 4.8.5 and Qt 4.8.2. In both cases happens more or less the same, there is no significant change in the memory consumption between both backends.
Comment 14 Emilio 2012-08-17 11:46:00 UTC
I've just compiled phonon-vlc-0.6 and I get the same result.
Comment 15 Myriam Schweingruber 2012-08-17 11:47:08 UTC
Thank you for the fast feedback.
Comment 16 Edward Hades 2012-08-18 12:27:31 UTC
Git commit 2d80e38d0dce4201e0c89b742f8a3f7267074f58 by Edward Hades.
Committed on 18/08/2012 at 14:24.
Pushed by hades into branch 'master'.

magnatune: flush database after commit

This makes mysql close unused tables and free corresponding caches. In
the case of Magnatune update this amounts to some 300M.

In the future, we would probably like to either limit Mysql memory usage
somehow, or at least ask upstream to implement some sort of adaptive
cache size selection, because users have reported OOM conditions on low
memory systems.

M  +1    -1    src/services/magnatune/MagnatuneDatabaseHandler.cpp

Comment 17 Edward Hades 2012-08-18 12:34:43 UTC
Dear Marcus and everyone affected,

The issue with Magnatune update lies deep within our internal database engine, which is Mysql. It eats this many memory for temporary cache. Unfortunately, it does not (apparently, as seen from this bug report) limit this memory to the total memory available. So, for now Magnatune updates may kill Amarok on low-memory machines.

We shall discuss possible solutions and fix the problem in the future. If you want, you may open a "limit memory usage for low-memory machines" feature request in order to track our progress.

However, for machines with sufficient memory, the problem reported in the bug has been solved: all the cache memory is released after the Magnatune update.