| Summary: | valgrind is unusably large when linked with lld | ||
|---|---|---|---|
| Product: | [Developer tools] valgrind | Reporter: | Matt Turner <mattst88> |
| Component: | general | Assignee: | Julian Seward <jseward> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | emaste, funous, mark, pjfloyd |
| Priority: | NOR | ||
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Platform: | Other | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | ||
| Sentry Crash Report: | |||
|
Description
Matt Turner
2021-06-23 05:26:32 UTC
(In reply to Matt Turner from comment #0) > [2] https://issuetracker.google.com/issues/191520718 That requires some kind of google account to view. Could you post your analysis somewhere public or in this bug report? Here is a copy/paste of the text.
Created issue.
tl;dr
ChromeOS's clang/lld pads out the .text section by the amount given by the -Ttext=... option. This produces enormous executables in the dev-util/valgrind package. gcc/ld does not do this on ChromeOS, nor does clang/lld on vanilla Gentoo.
Reproduction steps:
$ cat boom.c
int _start () { return 0; }
$ x86_64-cros-linux-gnu-gcc -static -nodefaultlibs -nostartfiles -Wl,-Ttext=0x58000000 boom.c -o boom
$ ls -lh boom
-rwxrwxr-x 1 msturner primarygroup 9.0K Jun 19 02:20 boom
$ x86_64-cros-linux-gnu-clang -static -nodefaultlibs -nostartfiles -Wl,-Ttext=0x58000000 boom.c -o boom
$ ls -lh boom
-rwxrwxr-x 1 msturner primarygroup 1.4G Jun 19 02:20 boom
Background
When built on ChromeOS with clang/lld, dev-util/valgrind takes far too much disk space, making it too large to deploy a DUT:
emerge-lakitu valgrind
[...]
* Final size of build directory: 14706448 KiB (14.0 GiB)
* Final size of installed tree: 14514812 KiB (13.8 GiB)
whereas when built with gcc/ld it takes a much more expected amount of space:
CC="x86_64-cros-linux-gnu-gcc" CXX="x86_64-cros-linux-gnu-g++" emerge-lakitu valgrind
[...]
* Final size of build directory: 401664 KiB (392.2 MiB)
* Final size of installed tree: 181876 KiB (177.6 MiB)
This is because a handful of static executables (e.g., memcheck-amd64-linux) end up being ~1.4GB each in the clang/lld build and only a few MB when built with gcc/ld.
Those static executables are linked with a linker option that positions the .text section start address at 0x58000000 (1.375GB). See this for rationale.
The configure script tests for and chooses -Ttext-segment=... if available and falls back to -Ttext=... otherwise. LLD does not support -Ttext-segment=...:
$ x86_64-cros-linux-gnu-ld.lld -Ttext-segment=0x58000000
ld.lld: error: -Ttext-segment is not supported. Use --image-base if you intend to set the base address
so it uses -Ttext=... instead.
readelf -a memcheck-amd64-linux on one of the clang/lld-produced 1.4GB files shows what's happening:
$ ls -lh memcheck-amd64-linux
-rwxrwxr-x 1 msturner portage 1.4G Jun 19 01:58 memcheck-amd64-linux
$ readelf -a memcheck-amd64-linux
[...]
Entry point address: 0x580ff894
Start of program headers: 64 (bytes into file)
Start of section headers: 1481854064 (bytes into file)
[...]
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .text PROGBITS 0000000058000000 57e00000
0000000000196db3 0000000000000000 AX 0 0 8
[ 2] .rodata PROGBITS 0000000058196dc0 57f96dc0
00000000000796d8 0000000000000000 AMS 0 0 16
whereas with gcc/ld.bfd:
$ ls -lh memcheck-amd64-linux
-rwxrwxr-x 1 msturner portage 12M Jun 19 01:53 memcheck-amd64-linux
$ readelf -a memcheck-amd64-linux
[...]
Entry point address: 0x580846c5
Start of program headers: 64 (bytes into file)
Start of section headers: 12045048 (bytes into file)
[...]
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .note.gnu.bu[...] NOTE 00000000580001c8 000001c8
0000000000000024 0000000000000000 A 0 0 4
[ 2] .text PROGBITS 0000000058001000 00001000
000000000016901a 0000000000000000 AX 0 0 16
[ 3] .rodata PROGBITS 000000005816b000 0016b000
000000000006c1e8 0000000000000000 A 0 0 32
That is, using -Ttext=0x58000000 with ChromeOS's lld seems to actually pad out the section headers by roughly 0x58000000 bytes! (See the Offset of the .text section and the Start of section headers)
Notably, using -Ttext=0x58000000 with gcc/ld.bfd does not do this and instead produces a reasonably-sized binary.
I tried reproducing this clang/lld behavior on a plain Gentoo install, and I could not, which leads me to wonder if ChromeOS's clang/lld is configured with some hardening option that causes this...
Modifying valgrind's configure to use --image-base=... instead of -Ttext=... seems to work and produce a reasonably-sized executables (even smaller than gcc/ld).
diff --git a/configure.ac b/configure.ac
index 4582fb5d0..86cd9176b 100755
--- a/configure.ac
+++ b/configure.ac
@@ -2657,7 +2657,7 @@ AC_LINK_IFELSE(
AC_MSG_RESULT([yes])
], [
linker_using_t_text="yes"
- AC_SUBST([FLAG_T_TEXT], ["-Ttext"])
+ AC_SUBST([FLAG_T_TEXT], ["--image-base"])
AC_MSG_RESULT([no])
])
CFLAGS=$safe_CFLAGS
---
* Final size of build directory: 308972 KiB (301.7 MiB)
* Final size of installed tree: 117356 KiB (114.6 MiB)
$ ls -lh memcheck-amd64-linux
-rwxrwxr-x 1 msturner portage 7.3M Jun 19 02:12 memcheck-amd64-linux
$ readelf -a memcheck-amd64-linux
[...]
Entry point address: 0x581792c4
Start of program headers: 64 (bytes into file)
Start of section headers: 7556672 (bytes into file)
[...]
Section Headers:$ readelf -a memcheck-amd64-linux
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .rodata PROGBITS 00000000580001d0 000001d0
00000000000796d8 0000000000000000 AMS 0 0 16
[ 2] .eh_frame_hdr PROGBITS 00000000580798a8 000798a8
0000000000000034 0000000000000000 A 0 0 4
[ 3] .eh_frame PROGBITS 00000000580798e0 000798e0
000000000000014c 0000000000000000 A 0 0 8
[ 4] .text PROGBITS 0000000058079a30 00079a30
0000000000196db3 0000000000000000 AX 0 0 8
Questions
Why does x86_64-cros-linux-gnu-clang behave this way?
Should valgrind's configure script test for and use --image-base=... if available?
ms...@google.com<ms...@google.com> #2Jun 21, 2021 10:52PM
The FreeBSD port of valgrind contains a patch that looks relevant and fixes the issue I see: https://github.com/paulfloyd/freebsd_valgrind/commit/9748df5f6a0cefd5b4a4beeeffe1d3668611e561
It contains a comment that seems to describe the problem somewhat:
+# - LLVM's ld.lld, for at least versions 8.0 (shipping with FreeBSD 12.1)
+# and 9.0 support the -Ttext option and behave as desired. As of
+# LLVM ld.lld version 10.0 a breaking change made -Ttext unusable,
+# however the --image-base option has the desired semantics.
+# It turns out that ld.lld has supported --image-base since at least
+# as far back as version 8.0.
I'll try to connect with the patch authors and see if I can help get this upstreamed.
(In reply to Mark Wielaard from comment #1) > (In reply to Matt Turner from comment #0) > > [2] https://issuetracker.google.com/issues/191520718 > > That requires some kind of google account to view. > Could you post your analysis somewhere public or in this bug report? Sorry about that, and thanks Paul for reposting. I wrote it in markdown, so here's a copy with the formatting: https://gist.github.com/mattst88/be75c31d0ec33d5cc51124625e410799 *** Bug 426135 has been marked as a duplicate of this bug. *** This should now be fixed with my recent series of commits to merge FreeBSD support. |