Bug 96228 - Running out of (internal) memory while loading shared libs
Summary: Running out of (internal) memory while loading shared libs
Status: RESOLVED DUPLICATE of bug 92071
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (show other bugs)
Version: 2.2.0
Platform: Compiled Sources Linux
: NOR crash
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-01-03 18:15 UTC by Wolfgang Wander
Modified: 2005-01-24 10:54 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Wolfgang Wander 2005-01-03 18:15:17 UTC
Version:           2.2.0 (using KDE KDE 3.2.1)
Installed from:    Compiled From Sources
Compiler:          gcc 3.3.4 
OS:                Linux

valgrind (since version 2.1) seems to have trouble to load executables with
large amounts of symbols.  In this examble it crashes while trying to load
shared libraries.

This is independent of the tools (tested only memcheck and addrcheck):

VG_(get_memory_from_mmap): newSuperblock's request for 1048576 bytes failed 12
VG_(get_memory_from_mmap): 164951173 bytes already allocated.

clearly 164MB should not be a reason to die for ENOMEM reasons.

Tracking down the problem it seems that valgrind's continued requests for
1MB chunks from newSuperblock

get_memory_from_mmap: 164951173 tot, 1048576 req = 0xBFE91000 .. 0xBFF90FFF, caller newSuperblock

drive the system out of memory in find_mmap_space

find_map_space: ret starts as 0xB0000000-0xB0102000 client=0
s->addr=0xB0000000 len=552960 (0xB0087000) ret=0xB0000000
s->addr=0xB0087000 len=4096 (0xB0088000) ret=0xB0087000
s->addr=0xB0088000 len=1417216 (0xB01E2000) ret=0xB0088000
s->addr=0xB01E3000 len=1048576 (0xB02E3000) ret=0xB01E2000
[...]
s->addr=0xBFE91000 len=1048576 (0xBFF91000) ret=0xBFE90000
s->addr=0xBFFFB000 len=20480 (0xC0000000) ret=0xBFF91000
  s->addr=0xFFFFE000 ->len=4096
find_map_space(0xB0000000, 1056768, 0) -> 0x0

VG_(get_memory_from_mmap): newSuperblock's request for 1048576 bytes failed 12
VG_(get_memory_from_mmap): 164951173 bytes already allocated.

Is there a clean way to lower kickstart_base (or increase valgrind_end) in the
memory layout to allow me to load my huge executables into the system?

    Thanks for any help or hint...
                Wolfgang
Comment 1 Wolfgang Wander 2005-01-03 20:42:15 UTC
I actually got it running my lowering KICKSTART_BASE in coregrind/Makefile.in to
7A000000 (from B0000000).  Is that the preferred solution and if so, would it
be worthwhile for me to consider a patch to valgrind for moving the mmaps for shared lib maps into the user space?
Comment 2 Tom Hughes 2005-01-03 20:49:14 UTC
In message <20050103194216.32034.qmail@ktown.kde.org> you wrote:

> I actually got it running my lowering KICKSTART_BASE in
> coregrind/Makefile.in to 7A000000 (from B0000000).  Is that the preferred
> solution and if so, would it be worthwhile for me to consider a patch to
> valgrind for moving the mmaps for shared lib maps into the user space?

What is "the user space" that you are talking about?

There are serious problems at the moment with memory space - lowering
the base like that helps you but hinders people with large amounts of
data as it restricts the amount of space available for the program being
run under valgrind.

The solution is two fold - one is 64 bit machines and the other is
an incremental reader for debug information.

The CVS code also tried to move valgrind itself up as high as possible
on systems which support position independent executables.

Tom

Comment 3 Wolfgang Wander 2005-01-03 22:11:01 UTC
Tom Hughes wrote

>What is "the user space" that you are talking about?
>  
>
 

Naively I was assuming I could re-call mmap from
within get_memory_from_mmap if I get a failure
but this time with VKI_MAP_CLIENT set to get
memory from the pool between between CLIENT_BASE
and client_end.  But this naive assumption comes from
the fact that I don't really understand the separation of
valgrind memory and client memory in the first place...

Learning ;-)

          Wolfgang

Comment 4 Dominik Stadler 2005-01-24 10:46:20 UTC
This bug sounds somewhat like a duplicate of Bug 92071.
Comment 5 Tom Hughes 2005-01-24 10:54:03 UTC
Indeed. I would also note that the reporters suggestion of retrying the map but from the client space is a bit of a bodge - it might work but defeat the object of separating valgrind's internal memory from the client's memory.

For debug reading that may not be too much of an issue as it will be released again before the client gets control I think. That would need to be handled at a higher level that get_memory_from_mmap however as that is probably also used for longer term things that shouldn't be in the client space.

*** This bug has been marked as a duplicate of 92071 ***