Bug 291129

How mant memory do you have?

4GB RAM on my machine.

That is the specified behaviour, stuff is left on cache so it is faster to revisit the pages, if that's not the behaviour you want choose a different Memory Usage setting in the performance configuration tab.

I'm sorry but your answer about default behaviour sounds not professional, or may be you don't realize the whole situation. 

My test machine use Arch and kde 4.8. I have a 4GB RAM.

Let's go to the "Okular Settings" to "Performance" tab. 
We see "Memory Usege" set to "Normal (default)", and later we have a discription: 
"A good compromise between memory usege and speed gain. Preload next page and boost searches. (For systems with 256MB of memory, typically.)"

And after scrolling book with 400 pages I heve memory consumption of 1380Mb. 
1380 >> 256 =) Ok. I understand that nobody changes this "help text" since KDE3 or somehow...

BUT the real thing is that opening few documents, lets say 3 or ethen 3 instance of the same book and scrolling till the end will end up with more than 4GB in Total. Not smart at all. 
But of couse ist fast. Very fast. It keep whole book, page by page in memory=)))

Ahh, the old "you are not professional" meme.

Ok, i'm not a professional, i'm just a person that has been working for KDE in my free time since 2003. If you want me to be a professional I will take your money deliver a badly coden software and do it late as real professionals i know do, is this a deal?

And yes, the text has not been updated, writing there a number there was not a good idea.

And yes they might be a bug in how we handle memory.

But without you giving me your exact hardware i know i can't fix it, i've been over that code lots of times and looks correct enough to me

Reopen the bug if you want, the end result will be the same, noone will work on it since caching works for us.

On the other hand if you that are the one having the problem have the skills to code a fix drop by the #okular IRC channel and we'll give you a hand in debugging. That would be professional on your side.

Well, I almost forgot this report opened by me.

Albert, I'm fine with the technical explanation and the rationale behind it. I just think it is not quite appropriate to be selected as the default mode. That is mainly about user expectation.

There are some applications that are well known for huge memory occupation: FireFox, VirtualBox, etc. Maybe I would also add acroread from Adobe. I would not be surprised if they make my system slow and would be careful about free memory when starting/using them.  But I just have never expected okular to be one of them. I guess other users might hold similar expectation and would be annoyed when they are hitten by this default mode.  They would then think okular is working in a buggy way and open bug report for it. Or worse, they would think okular is just "so crazy" and stop using it.

So my suggestion is change the existing description so that users understand its impact better, and/or use "low" as the default mode.

I didn’t meant that when i talked about "professional".
I think that Jekyll Wu better explained my point of view. 

I checked behavior for two position in okular "default" and "aggressive" and didn't found any difference. In both case it took about 1.4Gb for scrolled 400pages pdf. For the case of "Low memory" okular tooks about 250Mb for same activity. 
And the main thing -- I didn't found big difference between this two regimes. 
At "Low" a see only very short and fast rendering of pages while scrolling.

As said, you guys are free to debug the memory caching and total/free memory calculation code in document.cpp and find any bugs there (i'm of course not saying that code is bug free).

Okular uses memory if there unused one (why would you prefer memory doing nothing?) and frees memory if it is using more memory than the one that is actually free (thus we do not exhaust memory or monopolize it if someone starts to need it)

As I said before, I started 3 times exactly the same file, scroll over 400pages, and each gets about 1400mb, so ALL of my 4GB was used by 3 instance of okular.

When as you say "and frees memory if it is using more memory than the one that is
actually free". If any new application is going to be run, first of all it should ask okular to free some memory, or even go swapping to be able to run. 
Sorry, here is only theory, I didn't try to fill all of my memory with okular and after it trying to run anything else. 

By the way, I didn't see such problem in okular 4.4.5 and in 4.6.5. May be in 4.7.2 where was also no problem (not sure).

Yes, i read what you wrote, did you read what I wrote?

* There might be bugs, obviously
* If they are bugs i can't fix them since it works for me, so you are on your own

Can you at least check this idea:
old "Low" is missing and not possible to choose
old "Normal" is now "Low"
old "Aggressive" is now "Normal"
and for new "Aggressive" is again "Aggressive"

As I see no difference in behavior between "Aggressive" and "Normal", 
and you don't gonna look for bugs, may be at least just remove third choice and change text description.

The code hasn't changed for years.

I didn't know that.
In this case I see only two possible reasons: Qt or Distribution.

Hi,

I think I found a bug in DocumentPrivate::getFreeMemory().
I was programming on Gwenview which has a copied version of this function in its code.

When writing the output of getFreeMemory() to stdout via qDebug() I sometimes get values like 18446744073694777344 which suggest that there is an integer underflow. Here is an excerpt from cat /proc/meminfo shortly after that happened:
MemTotal:        2824516 kB
MemFree:          277580 kB
Buffers:           10292 kB
Cached:           169928 kB
SwapCached:        32984 kB
Active:          1733572 kB
Inactive:         546212 kB
Active(anon):    1663072 kB
Inactive(anon):   468348 kB
Active(file):      70500 kB
Inactive(file):    77864 kB
Unevictable:       11316 kB
Mlocked:           11316 kB
SwapTotal:       7814140 kB
SwapFree:        7334064 kB

When I understand the function properly it is calculation the free memory as:
MemFree + Buffers + Cached + SwapFree - SwapTotal
which in this case would result in a negative value of -22 276 kB which can't be returned as qulonglong.

Git commit 444e6b7b19bee5285f39a44007cf00f6e0b235d2 by Albert Astals Cid.
Committed on 20/06/2012 at 00:33.
Pushed by aacid into branch 'master'.

Fix underflow in getFreeMemory()

It actually serves three purposes:
a) Make sure all the values are there (this should be always true, but doesn't hurt making sure) because if SwapFree was there but SwapTotal was not, it'd be a mess
b) add up things in order so we don't underflow, currently the code did process stuff as it came in the file, and it happens that SwapTotal appears before SwapFree in /proc/meminfo so it actually did  "MemFree:" + "Buffers:" + "Cached:" - "SwapTotal:" + "SwapFree:", which can underflow if "MemFree:" + "Buffers:" + "Cached:" < "SwapTotal:"
c) Do not underflow at all, so if  "MemFree:" + "Buffers:" + "Cached:" +  "SwapFree:" < "SwapTotal:" we return 0 correctly not a zillion of free memory

Aurélien you might want to update gwenviews copy of this code (there's a few other bugfixes we did a while ago you didn't update either)

CCMAIL: agateau@kde.org

M  +22   -7    core/document.cpp

http://commits.kde.org/okular/444e6b7b19bee5285f39a44007cf00f6e0b235d2

Git commit 5fb937a46aaf4e531ada816692b1e47fc95b20b0 by Albert Astals Cid.
Committed on 20/06/2012 at 00:33.
Pushed by aacid into branch 'KDE/4.8'.

Fix underflow in getFreeMemory()

It actually serves three purposes:
a) Make sure all the values are there (this should be always true, but doesn't hurt making sure) because if SwapFree was there but SwapTotal was not, it'd be a mess
b) add up things in order so we don't underflow, currently the code did process stuff as it came in the file, and it happens that SwapTotal appears before SwapFree in /proc/meminfo so it actually did  "MemFree:" + "Buffers:" + "Cached:" - "SwapTotal:" + "SwapFree:", which can underflow if "MemFree:" + "Buffers:" + "Cached:" < "SwapTotal:"
c) Do not underflow at all, so if  "MemFree:" + "Buffers:" + "Cached:" +  "SwapFree:" < "SwapTotal:" we return 0 correctly not a zillion of free memory

Aurélien you might want to update gwenviews copy of this code (there's a few other bugfixes we did a while ago you didn't update either)

CCMAIL: agateau@kde.org
(cherry picked from commit 444e6b7b19bee5285f39a44007cf00f6e0b235d2)

M  +22   -7    core/document.cpp

http://commits.kde.org/okular/5fb937a46aaf4e531ada816692b1e47fc95b20b0

(In reply to comment #16)

Albert,
I didn't understand why swapped memory is considered for free memory calculation in Linux. After having a look at the FreeBSD and Windows part of getFreeMemory(), which are just returning the free physical memory as it seems, I'm totally confused. Could you explain this briefly, please?

The swapped memory is "substracted" from the free memory because we don't want to prevent swapping in of any program that might be swapped out to be slow or to end up putting okular stuff in the swap.

But to be honest i did not write that code and i'm not sure if that's the right thing to do, all I just did was to fix the technical problems.

And as always everyone says you should not do memry managemtn in the program by yourself so for a lot of people "we are doing it wrong", if you have a much better suggestion and a real good rationale behind it i'll be happy to hear it

(In reply to comment #18)
> The swapped memory is "substracted" from the free memory because we don't
> want to prevent swapping in of any program that might be swapped out to be
> slow or to end up putting okular stuff in the swap.

I understand, but in this case SwapCached should also be considered then. According to [1] SwapCached is "Memory that once was swapped out, is swapped back in but still also is in the swapfile (if memory is needed it doesn't need to be swapped out AGAIN because it is already in the swapfile. This saves I/O)". So SwapCached could be added to memoryFree because it's already in physical memory.

But I would prefer to change the behavior of getFreeMemory:
Instead of a program that is swapping in, the memory could also been taken away by a program that is just starting up. To detect this okular is already polling the memory usage every 2 s (if I get this right). So why can't we detect programs that are swapped in, this way? I would propose to just poll the physical memory usage (perhaps faster) and check if (freePhysicalMemory < fraction*totalPhysicalMemory) with fraction ~0.5 or so. If thats the case then we free some memory.

And finally: Shouldn't getFreeMemory behave consistently on all platforms? (BTW: Is Mac OS X missing?)


Another thing related to memory usage:
In DocumentPrivate::cleanupPixmapMemory when set to Greedy:
memoryLimit = qMin( qMax( freeMemory, getTotalMemory()/2 ), freeMemory+freeSwap );

Let's assume freeSwap = 10 GB, freeMemory = 1 GB and totalMemory = 8 GB, then memoryLimit would be totalMemory/2 (= 4 GB) and would result in swapping? Or do I understand this wrong?



[1]: https://lwn.net/Articles/28345/

(In reply to comment #18)
> And as always everyone says you should not do memry managemtn in the program
> by yourself so for a lot of people "we are doing it wrong",

I don't know how to do it right. Don't use memory? ;-)

If you know how to implement the MacOSX version we'll take the patch ;-)

About the greedy stuff, Yes, it would result in us kicking out of the memory some oher program to the swap and using the memory, that's why it's called greedy ;-)

OK, but then there should be a big warning like "might make your system slow and unresponsive".
And in doc/index.docbook this should also be less misunderstanding:
"Use Greedy profile to preload all pages without risk of system memory overfull..."

You are deviating the topic :D

If you feel the doc is wrong, please open a bug about that, and no, i don't think there is a need for a big warning about greedy.

I had the same issue: default "normal" setting used to be too greedy and sometimes freezed my system. This could be a little annoying for less skilled users who typically don't proofread program settings before using them.

Anyway, setting the radio button to "minimal" does the trick for me.

*** Removed by KDE Sysadmin due to violation of abuse policies ***