Bug 313267 - Adding MIPS64/Linux port to Valgrind
Summary: Adding MIPS64/Linux port to Valgrind
Status: RESOLVED FIXED
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (show other bugs)
Version: 3.9.0.SVN
Platform: Debian unstable Linux
: NOR normal
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-14 20:14 UTC by Petar Jovanovic
Modified: 2013-09-20 00:16 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
patch for VEX (r2633) - MIPS64/Linux (448.95 KB, patch)
2013-01-14 20:51 UTC, Petar Jovanovic
Details
patch for Valgrind (r13228) - MIPS64/Linux (309.09 KB, patch)
2013-01-14 20:53 UTC, Petar Jovanovic
Details
patch for VEX (r2686) - MIPS64/Linux (461.80 KB, patch)
2013-02-26 22:48 UTC, Petar Jovanovic
Details
patch for Valgrind (r13290) - MIPS64/Linux (306.49 KB, patch)
2013-02-26 22:53 UTC, Petar Jovanovic
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Petar Jovanovic 2013-01-14 20:14:25 UTC
Here is an issue to track down patches and review of MIPS64 support.

Reproducible: Always
Comment 1 Petar Jovanovic 2013-01-14 20:51:39 UTC
Created attachment 76471 [details]
patch for VEX (r2633) - MIPS64/Linux

Apply against r2633.
Comment 2 Petar Jovanovic 2013-01-14 20:53:42 UTC
Created attachment 76472 [details]
patch for Valgrind (r13228) - MIPS64/Linux

Apply against r13228.
Comment 3 Petar Jovanovic 2013-01-14 20:57:29 UTC
Two patches have been attached to the issue:
- valgrind_mips64_r13228_2633_VEX.diff, and
- valgrind_mips64_r13228_2633.diff.

The patches should be applied against r13228 (external 2633) revision from
the main Valgrind trunk.

If you want to apply the patch and build Valgrind natively, here are brief
instructions. Before starting, make sure you actually do have operating system
with MIPS64 binaries with properly configured toolchain.

$ svn co -r13228 svn://svn.valgrind.org/valgrind/trunk r13228
$ cd r13228/VEX
$ svn up -r2633
$ patch -p0 < ../../valgrind_mips64_r13228_2633_VEX.diff
$ cd ..
$ patch -p0 < ../valgrind_mips64_r13228_2633.diff
$ ./autogen.sh
$ ./configure
$ make
$ make install

You can also use the cross-toolchain, if you prefer it.
Comment 4 Petar Jovanovic 2013-01-14 21:00:53 UTC
Additional note: the patches add support for MIPS64 Little-Endian.
Support for Big-Endian will follow later.
Comment 5 Junaid 2013-02-19 08:05:42 UTC
 svn co -r13228 svn://svn.valgrind.org/valgrind/trunk r13228

It is not accessible/working. 
Error:
svn: E670005: Unable to connect to a repository at URL 'svn://svn.valgrind.org/valgrind/trunk'
svn: E670005: Unknown hostname 'svn.valgrind.org'

Currently i am connected to internet via company network which have proxy server. Is the unaccessibility is due to proxy server?
I am using --config-option servers:global:http-proxy-host=[Network Proxy] --config-option servers:global:http-proxy-port=8080

Can you please tell me any way to get this. Or
Should i try it using direct connection?
Comment 6 Petar Jovanovic 2013-02-19 12:47:14 UTC
You may want to enable proxy in:

 ~/.subversion/servers

Look for something like:
[global]
#http-proxy-host = myproxy
#http-proxy-port = 8080

Remove '#' and set the correct values. That should do the trick.
Comment 7 Petar Jovanovic 2013-02-19 13:09:06 UTC
Now that I said the above, I forgot that svn co for Valgrind is done through
svn:// which makes any changes to http-proxy-host irrelevant for this.

There are couple of ways to get around this, but it may be easier for
you to ask your company IT department to enable svn port for checkout.
Comment 8 Tom Hughes 2013-02-19 13:20:05 UTC
Which is still not very helpful advice when the problem is that he can't resolve the hostname... If that persists then it is his DNS setup he needs to investigate first, not whether the svn port is blocked.
Comment 9 Petar Jovanovic 2013-02-19 13:38:29 UTC
well, I don't have a helpful advice how to debug corporate network and its DNS,
etc. I was more saying that he should bring the issue to the attention of IT
department in his company, as I would not be surprised if the error output he
gets is a misleading one.

Potentially helpful advice could be that he tries to use IP address itself, and
see what happens, something like:

svn co svn://178.250.76.52/valgrind/trunk
Comment 10 Junaid 2013-02-19 17:38:40 UTC
Well thanks for your help. I have done that with direct connection and i shall inform the IT department about svn port.
Comment 11 Rakesh Garg 2013-02-20 17:50:40 UTC
I am trying to validate Valgrind on MIPS64 LE target and is observing these errors:

--1244-- core : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- tool : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- dinfo : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- (null) : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- demangle: 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- exectxt : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- errors : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
--1244-- ttaux : 0/ 0 max/curr mmap'd, 0/0 unsplit/split sb unmmap'd, 0/ 0 max/curr, B
==1244==
==1244== Valgrind's memory management: out of memory:
==1244== newSuperblock's request for 4194304 bytes failed.
==1244== 13697024 bytes have already been allocated.
==1244== Valgrind cannot continue. Sorry.
==1244==
==1244== There are several possible reasons for this.
==1244== - You have some kind of memory limit in place. Look at the
==1244== output of 'ulimit -a'. Is there a limit on the size of
==1244== virtual memory or address space?
==1244== - You have run out of swap space.
==1244== - Valgrind has a bug. If you think this is the case or you are
==1244== not sure, please let us know and we'll try to fix it.
==1244== Please note that programs can take substantially more memory than
==1244== normal when running under Valgrind tools, eg. up to twice or
==1244== more, depending on the tool. On a 64-bit machine, Valgrind
==1244== should be able to make use of up 32GB memory. On a 32-bit
==1244== machine, Valgrind should be able to use all the memory available
==1244== to a single process, up to 4GB if that's how you have your
==1244== kernel configured. Most 32-bit Linux setups allow a maximum of
==1244== 3GB per process.
==1244==
==1244== Whatever the reason, Valgrind cannot continue.Sorry.

I have configured my cross-toolchains appropriately before building Valgrind for my target and am using these commands:
$ svn co -r13228 svn://svn.valgrind.org/valgrind/trunk valgrind_r13228
$ cd valgrind_r13228/VEX
$ svn up -r2633
$ patch -p0 < ../../valgrind_mips64_r13228_2633_VEX.diff
$ cd ..
$ patch -p0 < ../valgrind_mips64_r13228_2633.diff
$ ./autogen.sh
$ ./configure  --host=mips64el-linux-gnu --with-pagesize=64 CFLAGS="-EL -mips64 –mplt –msym32 –mhard-float" --prefix=/home/usr/valgrind_r13228
$ make
$ make install

From the above output it seems that it has failed to allocate ~4MB of memory whereas it has already allocated ~13MB previously.

This issue does not seem to be related to unavailability of memory as I've tried test programs which consumes even more than 120MB of memory. 

Kindly let me know if I am missing something in the steps mentioned above or provide me some pointers to debug this issue further.

Thanks in advance.
Comment 12 Petar Jovanovic 2013-02-21 13:36:49 UTC
For everyone's notice, as communicated in a different thread with
Ragesh, solution for his environment was to ensure the ABI is n64 by
adding extra "–mabi=64" to CFLAGS.
Comment 13 Petar Jovanovic 2013-02-26 22:48:57 UTC
Created attachment 77604 [details]
patch for VEX (r2686) - MIPS64/Linux

Apply against r2686.
Comment 14 Petar Jovanovic 2013-02-26 22:53:21 UTC
Created attachment 77605 [details]
patch for Valgrind (r13290) - MIPS64/Linux

Apply against r13290.
Comment 15 Petar Jovanovic 2013-02-26 23:01:19 UTC
Note to Valgrind MIPS64 users - the patches have just been updated.

Two new patches have been attached to the issue:
- valgrind_mips64_r13290_2686_VEX.diff, and
- valgrind_mips64_r13290_2686.diff.

The patches should be applied against r13290 (external 2686) revision from
the main Valgrind trunk.

This makes the previous patches obsolete, so do not use the older ones.

If you want to apply the patches and build Valgrind natively, here are brief
instructions. Make sure first you have an operating system with MIPS64 (n64 abi)
binaries with properly configured toolchain.

$ svn co -r13290 svn://svn.valgrind.org/valgrind/trunk r13290
$ cd r13290/VEX
$ svn up -r2686
$ patch -p1 < ../../valgrind_mips64_r13290_2686_VEX.diff
$ cd ..
$ patch -p0 < ../valgrind_mips64_r13290_2686.diff
$ ./autogen.sh
$ ./configure
$ make
$ make install

You can also use the cross-toolchain, if you prefer it.
Comment 16 Julian Seward 2013-02-27 15:31:20 UTC
(In reply to comment #14)
> Created attachment 77605 [details]
> patch for Valgrind (r13290) - MIPS64/Linux

bug313267c14-mips64-val.diff (valgrind side diffs):

Does the trunk still work and have unchanged regtest results on
x86_64-linux, with this patch applied?

--------

    172 +++ coregrind/pub_core_trampoline.h     (working copy)
    173 @@ -133,6 +133,12 @@
    174  extern UInt  VG_(mips32_linux_REDIR_FOR_strlen)( void* );
    175  #endif
    176  
    177 +#if defined(VGP_mips64_linux)
    178 +extern Addr  VG_(mips64_linux_SUBST_FOR_sigreturn);
    179 +extern Addr  VG_(mips64_linux_SUBST_FOR_rt_sigreturn);
    180 +extern UInt  VG_(mips64_linux_REDIR_FOR_strlen)( void* );
    181 +#endif

Do all of these functions actually exist in the assembly?  It has
happened before now that the header claims the existence of
non-existent functions.
Seems like VG_(mips64_linux_SUBST_FOR_sigreturn) doesn't exist.

--------

   2334 Index: coregrind/m_debuginfo/readdwarf.c
   2338 @@ -527,9 +527,9 @@
   2339        goto out;
   2340     }
   2341  
   2342 -   info.li_header_length = ui->dw64 ? ML_(read_ULong)(external) 
   2343 +   info.li_header_length = is64 ? ML_(read_ULong)(external) 
   2344                                      : (ULong)(ML_(read_UInt)(external));
   2345 -   external += ui->dw64 ? 8 : 4;
   2346 +   external += is64 ? 8 : 4;

The change from ui->dw64 to is64: why is that changed?  That does not
look correct to me.

--------

   5079 Index: include/vki/vki-posixtypes-mips64-linux.h
   5121 +#if (_MIPS_SZLONG == 64)

Is this correct?  Is _MIPS_SZLONG set somewhere?


   6170 Index: include/valgrind.h
   6198 -                     : "cc","memory", "t3", "t4");                \
   6199 +                     : "$11", "$12");                             \
   6207 -                     : "cc", "memory" , "t3"                      \
   6208 +                     : "$11"                                      \

Are you sure that removing cc and memory from these clobber lists is
safe?
Comment 17 Petar Jovanovic 2013-02-27 17:03:44 UTC
(In reply to comment #16)
> (In reply to comment #14)
> > Created attachment 77605 [details]
> > patch for Valgrind (r13290) - MIPS64/Linux
> 
> bug313267c14-mips64-val.diff (valgrind side diffs):
> 
> Does the trunk still work and have unchanged regtest results on
> x86_64-linux, with this patch applied?

The results are unchanged. 

$ nice make regtest -j4

== 648 tests, 27 stderr failures, 17 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==

on 

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.1 LTS
Release:        12.04
Codename:       precise

> 
> --------
> 
>     172 +++ coregrind/pub_core_trampoline.h     (working copy)
>     173 @@ -133,6 +133,12 @@
>     174  extern UInt  VG_(mips32_linux_REDIR_FOR_strlen)( void* );
>     175  #endif
>     176  
>     177 +#if defined(VGP_mips64_linux)
>     178 +extern Addr  VG_(mips64_linux_SUBST_FOR_sigreturn);
>     179 +extern Addr  VG_(mips64_linux_SUBST_FOR_rt_sigreturn);
>     180 +extern UInt  VG_(mips64_linux_REDIR_FOR_strlen)( void* );
>     181 +#endif
> 
> Do all of these functions actually exist in the assembly?  It has
> happened before now that the header claims the existence of
> non-existent functions.
> Seems like VG_(mips64_linux_SUBST_FOR_sigreturn) doesn't exist.
> 
> --------

mips64_linux_SUBST_FOR_sigreturn does not exists, the declaration should be removed.
Other two functions exist (coregrind/m_trampoline.S).

> 
>    2334 Index: coregrind/m_debuginfo/readdwarf.c
>    2338 @@ -527,9 +527,9 @@
>    2339        goto out;
>    2340     }
>    2341  
>    2342 -   info.li_header_length = ui->dw64 ? ML_(read_ULong)(external) 
>    2343 +   info.li_header_length = is64 ? ML_(read_ULong)(external) 
>    2344                                      :
> (ULong)(ML_(read_UInt)(external));
>    2345 -   external += ui->dw64 ? 8 : 4;
>    2346 +   external += is64 ? 8 : 4;
> 
> The change from ui->dw64 to is64: why is that changed?  That does not
> look correct to me.

We had an issue with one of the tests (memcheck/tests/dw4).
I suggest we revert this part of the change, and address/discuss this issue
separately.

> 
> --------
> 
>    5079 Index: include/vki/vki-posixtypes-mips64-linux.h
>    5121 +#if (_MIPS_SZLONG == 64)
> 
> Is this correct?  Is _MIPS_SZLONG set somewhere?

Yes, GCC defines it.
> 
> 
>    6170 Index: include/valgrind.h
>    6198 -                     : "cc","memory", "t3", "t4");                \
>    6199 +                     : "$11", "$12");                             \
>    6207 -                     : "cc", "memory" , "t3"                      \
>    6208 +                     : "$11"                                      \
> 
> Are you sure that removing cc and memory from these clobber lists is
> safe?

Yes. The block does not access memory, and having "cc" in the list of clobbered
registers is not meaningful for MIPS.
Comment 18 Julian Seward 2013-02-27 18:40:22 UTC
(In reply to comment #13)
> Created attachment 77604 [details]
> patch for VEX (r2686) - MIPS64/Linux

Looks OK -- I looked though it, but no comments.

OK to commit once you make the changes discussed in c16/c17
(removing the is64 change, and the redundant function prototype.)
Comment 19 Petar Jovanovic 2013-02-28 00:18:02 UTC
Changes committed as r13292 (Valgrind) and r2687 (VEX).
Comment 20 Petar Jovanovic 2013-03-01 18:05:47 UTC
(In reply to comment #16)
> (In reply to comment #14)
> --------
> 
>    2334 Index: coregrind/m_debuginfo/readdwarf.c
>    2338 @@ -527,9 +527,9 @@
>    2339        goto out;
>    2340     }
>    2341  
>    2342 -   info.li_header_length = ui->dw64 ? ML_(read_ULong)(external) 
>    2343 +   info.li_header_length = is64 ? ML_(read_ULong)(external) 
>    2344                                      :
> (ULong)(ML_(read_UInt)(external));
>    2345 -   external += ui->dw64 ? 8 : 4;
>    2346 +   external += is64 ? 8 : 4;
> 
> The change from ui->dw64 to is64: why is that changed?  That does not
> look correct to me.
> 
> --------

After second look, the change above still seems fine. It looks like we are
iterating over external line info using offsets calculated based on dw64 that
we got from .debug_info rather than based on is64 which is what we got from
.debug_line. This does not seem correct.

The change above is similar to what decode_line_info in dwarf2.c in gdb does.
Comment 21 Rakesh Garg 2013-03-21 07:03:27 UTC
I’m observing some unexpected behavior while using the valgrind option “log-socket”. Please find the more details for the same below:

On a remote linux machine, I ran the following command:
#> valgrind-listener 12345

On MIPS target board, I ran the following command:
#> valgrind --log-socket=<remote-linux-machine-ip>:12345 ./test.out

Now my expectation is that the valgrind commentary should be sent over the network to the remote linux machine that I’ve specified in the command, but that is not the case. Instead the commentary is printed on the target board itself and some initial logs are missed. Nothing is printed on the remote linux machine.

While debugging the issue further using wireshark, I found that the valgrind is sending the commentary via UDP packets whereas the valgrind-listener is expecting the same via TCP packets.

However, this option is working fine if I run valgrind on x86 linux machine.
Comment 22 Petar Jovanovic 2013-04-04 10:58:56 UTC
(In reply to comment #21)
> I’m observing some unexpected behavior while using the valgrind option
> “log-socket”. Please find the more details for the same below:
> 
> On a remote linux machine, I ran the following command:
> #> valgrind-listener 12345
> 
> On MIPS target board, I ran the following command:
> #> valgrind --log-socket=<remote-linux-machine-ip>:12345 ./test.out
> 
> Now my expectation is that the valgrind commentary should be sent over the
> network to the remote linux machine that I’ve specified in the command, but
> that is not the case. Instead the commentary is printed on the target board
> itself and some initial logs are missed. Nothing is printed on the remote
> linux machine.
> 
> While debugging the issue further using wireshark, I found that the valgrind
> is sending the commentary via UDP packets whereas the valgrind-listener is
> expecting the same via TCP packets.
> 
> However, this option is working fine if I run valgrind on x86 linux machine.

Fix committed in r13359.
Comment 23 Mark Wielaard 2013-04-19 13:00:58 UTC
(In reply to comment #20) above still seems fine. It looks like we are
> iterating over external line info using offsets calculated based on dw64 that
> we got from .debug_info rather than based on is64 which is what we got from
> .debug_line. This does not seem correct.

I agree. Using ui->dw64 here is wrong. It should use is64 as in the proposed patch.
Comment 24 Petar Jovanovic 2013-04-19 15:26:01 UTC
(In reply to comment #23)
> (In reply to comment #20) above still seems fine. It looks like we are
> > iterating over external line info using offsets calculated based on dw64 that
> > we got from .debug_info rather than based on is64 which is what we got from
> > .debug_line. This does not seem correct.
> 
> I agree. Using ui->dw64 here is wrong. It should use is64 as in the proposed
> patch.

Committed in r13373.