Bug 509406 - FreeBSD 15 issues
Summary: FreeBSD 15 issues
Status: REPORTED
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (other bugs)
Version First Reported In: 3.25 GIT
Platform: Other FreeBSD
: NOR normal
Target Milestone: ---
Assignee: Paul Floyd
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-09-12 05:51 UTC by Paul Floyd
Modified: 2025-10-06 20:16 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Floyd 2025-09-12 05:51:08 UTC
FreeBSD 15 has started ALPHA releases:
https://www.freebsd.org/releases/15.0R/schedule/

I'm seeing a couple of failures in a VM.

1. ./memcheck/tests/freebsd/setcred

Has an extra uninit error
--- setcred.stderr.exp  2025-09-12 09:11:53.900540000 +0200
+++ setcred.stderr.out  2025-09-12 09:18:27.818744000 +0200
@@ -20,6 +20,10 @@
    by 0x........: main (setcred.cpp:26)
  Address 0x........ is not stack'd, malloc'd or (recently) free'd
 
+Conditional jump or move depends on uninitialised value(s)
+   at 0x........: setcred (in /...libc...)
+   by 0x........: main (setcred.cpp:26)
+
 Syscall param setcred(wcred) points to uninitialised byte(s)
    at 0x........: setcred (in /...libc...)
    by 0x........: main (setcred.cpp:31)

Line 26 is
25   // uninit
26   setcred(flags2, (struct setcred*)x0, size1+x0);

It seems to be the last argument that is causing the problem.

2. none/tests/freebsd/bug452274
This testcase is hanging.

vgstack 58525
Thread 1 (Thread 101470 (tid 1 VgTs_Runnable)):
#0  _write () at _write.S:4
#1  0x0000000000201800 in main (argc=1, argv=0x1fe00009c8) at bug452274.c:31

int main(int argc, char *argv[]) {
  setup_timer();
  while (ticks < 100) {
    write(2, " ", 1);
  } 
}

It looks like the timer isn't firing (or the signal handler not running) and the loop above is busy writing to stderr all of the time.

3. none/tests/freebsd/proc_pid_file
+Assertion failed: (rp == resolvedPath2), function main, file proc_pid_file.cpp, line 20.

This one is easy. It's a new-ish test that requires /proc, which I don't have on my testing FreeBSD 15 VM.
Fixed.
Comment 1 Paul Floyd 2025-09-14 08:26:21 UTC
For the setcred fail, the full extra error is

==9528== Conditional jump or move depends on uninitialised value(s)
==9528==    at 0x4D2CC8A: setcred (in /lib/libsys.so.7)
==9528==    by 0x201849: main (setcred.cpp:26)
==9528==  Uninitialised value was created by a heap allocation
==9528==    at 0x484E224: malloc (vg_replace_malloc.c:451)
==9528==    by 0x2017D8: main (setcred.cpp:9)

And it looks like the conditional error is when checking the carry flag after the syscall. Strange.
Comment 2 Paul Floyd 2025-09-16 18:09:23 UTC
For bug452274 it looks like the interval timer has conked out or just isn't getting scheduled.

I can reproduce a hang if I run the TC outside of Valgrind, something like

 /usr/bin/time sh -c "none/tests/freebsd/bug452274  > /dev/null 2>&1"

Without the redirection (so going to a terminal over ssh) then the TC runs about 5x slower than natively.

I'll wait and see if the problem persists when I upgrade my main macchine to 15.0.
Comment 3 Paul Floyd 2025-09-16 19:47:24 UTC
On arm64 I get

./drd/tests/fork-parallel.stderr.diff

Needs a new suppression?

+Thread 3:
+Conflicting store by thread 3 at 0x........ size 1
+   at 0x........: ??? (in /...libc...)
+   by 0x........: ??? (in /...libc...)
+   by 0x........: ??? (in /...libc...)
+   by 0x........: _malloc_postfork (in /...libc...)
+   by 0x........: start_thread
+   by 0x........: start_thread
+   by 0x........: startproc (fork.c:?)
+   by 0x........: vgDrd_thread_wrapper (drd_pthread_intercepts.c:?)
+   by 0x........: start_thread
+   by 0x........: pthread_crestart_thread
+Allocation context: unknown.
+


./gdbserver_tests/mcinvokeWS.stdoutB.diff
./gdbserver_tests/mcinvokeWS.stderrB.diff

No output

./gdbserver_tests/nlcontrolc.stderr.diff
./gdbserver_tests/nlcontrolc.stdoutB.diff
./gdbserver_tests/nlsigvgdb.stderrB.diff
./gdbserver_tests/nlvgdbsigqueue.stderr.diff
./gdbserver_tests/nlvgdbsigqueue.stdoutB.diff

+valgrind: m_gdbserver/m_gdbserver.c:881 (void vgPlain_invoke_gdbserver(int)): Assertion 'check == 0x8BADF00D' failed.

./memcheck/tests/freebsd/pdfork_pdkill.stderr.diff

+Conditional jump or move depends on uninitialised value(s)
+   ...
+   by 0x........: main (pdfork_pdkill.c:105)
+

       pdfork(&anotherfd, badflag);

Strange.

./memcheck/tests/freebsd/scalar.stderr.diff

+Syscall param inotify_add_watch_at(mask) contains uninitialised byte(s)
+   ...
+

That's my fault. The syscall has 4 args and I only put 3 in scalar.c. Fixed.

./memcheck/tests/freebsd/timing_safe.stderr.diff

Several changes. Some missing and some changed errors. Not sure what is going on. Maybe on amd64 the compiler is replacing timingsafe_memcmp calls with code.

./none/tests/freebsd/auxv.stderr.diff
./none/tests/freebsd/auxv_script.stderr.diff

#define AT_CHERI_STATS  37      /* Reserved */
#define AT_HWCAP3       38      /* CPU feature flags 3. */
#define AT_HWCAP4       39      /* CPU feature flags 4. *

^^^ look like new entries that need to be handled
Comment 4 Paul Floyd 2025-09-17 19:31:23 UTC
Added a suppression for the DRD failure on arm64.
Updated the none auxv tests so that the guest runs first to generate the expexted. That should make it more reliable and only break when new AT_ items are added to auxv.

On amd64 I now get
./gdbserver_tests/nlcontrolc.stderr.diff
^^^ error message from select
./gdbserver_tests/nlvgdbsigqueue.stdoutB.diff
./gdbserver_tests/nlvgdbsigqueue.stderr.diff
^^^ simulate_control_c could not determine the vgdb pid
./memcheck/tests/freebsd/setcred.stderr.diff
^^^ as before, really annoying

On arm64
./gdbserver_tests/mcinvokeWS.stdoutB.diff
./gdbserver_tests/mcinvokeWS.stderrB.diff
./gdbserver_tests/nlcontrolc.stderr.diff
./gdbserver_tests/nlcontrolc.stdoutB.diff
./gdbserver_tests/nlsigvgdb.stderrB.diff
./gdbserver_tests/nlvgdbsigqueue.stderr.diff
./gdbserver_tests/nlvgdbsigqueue.stdoutB.diff
^^^ as before
./memcheck/tests/freebsd/timing_safe.stderr.diff
^^^ another annoying one
./memcheck/tests/thread_alloca.stderr.diff
^^^ has always been a bit flaky, now failing consistently
Comment 5 Paul Floyd 2025-09-17 19:49:37 UTC
I just looked at the gdbserver failures. In 2 of the cases (and probably the third, just not in the diffs) there is an assert firing.

How this is supposed to work:

invoker_invoke_gdbserver uses ptrace to modify the running Valgrind tool execution context. For arm64 the relevant bit is

      reg_mod.x[0] = check;
      reg_mod.sp = sp;
      reg_mod.elr = shared64->invoke_gdbserver;
      /* put NULL return address in Link Register */
      reg_mod.lr = bad_return;

So that's setting up the stack pointer, program counter (same as exception link register, dunno why), a phoney return address (s'pose the code wil get back via longjmp hyperspace) and in the x0 register the value of 'check', 0x8BADF00D.

On the other side, void VG_(invoke_gdbserver) ( int check ) contains

   vg_assert (check == 0x8BADF00D);

and that assert is failing. If I just comment out the assert then all the gdbserver tests pass on arm64.

I did a printf of the value of 'check' and it was 4.

There are only two things that I can see as possibilities. 

1.    if (ptrace(PT_SETREGS, pid, (caddr_t)regs, 0) < 0) {
^^^ if this has changed in some way in FreeBSD 15. I need to run the tests with -d to get traces for that.

2. Some compiler oddity at the receiving end. What I might be able to do there is to put a sleep in the code and then attach gdb to try to see what is happening.
Comment 6 Paul Floyd 2025-09-22 20:06:29 UTC
The first problem in timing_safe is that timingsafe_memcmp isn't getting redirected. It isn't in the names that generate_and_add_actives processes.
Comment 7 Paul Floyd 2025-09-25 19:48:38 UTC
The arm64 issue with timingsafe_memcmp is a FreeBSD bug. They use macros for the begin and end of asm functions. Looks like a copy and paste error, timingsafe_memcmp ends with END(timingsafe_bcmp).

See
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=289845
Comment 8 Paul Floyd 2025-10-03 19:03:46 UTC
Using clang19 (same as default FreeBSD 15 compiler)  on FreeBSD 14.2 didn't change anything.
Comment 9 Paul Floyd 2025-10-04 07:38:39 UTC
I've modified vgdb on arm64 to print out the first 10 32bit words from invoker_invoke_gdbserver using ptrace:

^CDEBUG: invoker_invoke_gdbserver 10 32bit words of instructions
a9be7bfd
f9000bf3
910003fd
529e01a8
72b175a8
6b08001f
54000c61
f0008bf3
b9425a68
350004e8

According to objdump that should be

Disassembly of section .text:

0000000000000e48 <vgPlain_invoke_gdbserver>:

/* Using ptrace calls, vgdb will force an invocation of gdbserver.
   VG_(invoke_gdbserver) is the entry point called through the
   vgdb ptrace technique. */
void VG_(invoke_gdbserver) ( int check )
{
     e48:       a9be7bfd        stp     x29, x30, [sp, #-32]!
     e4c:       f9000bf3        str     x19, [sp, #16]
     e50:       910003fd        mov     x29, sp
     e54:       529e01a8        mov     w8, #0xf00d                     // #61453
     e58:       72b175a8        movk    w8, #0x8bad, lsl #16
      gdbserver. Otherwise, we let the valgrind scheduler invoke
      gdbserver at the next poll.  This poll will be made very soon
      thanks to a call to VG_(force_vgdb_poll). */
   int n_tid, vgdb_interrupted_tid_local = 0;

   vg_assert (check == 0x8BADF00D);
     e5c:       6b08001f        cmp     w0, w8
     e60:       54000c61        b.ne    fec <vgPlain_invoke_gdbserver+0x1a4>  // b.any

   if (busy) {
     e64:       90000013        adrp    x19, 0 <vgPlain_ppPointKind>
     e68:       b9400268        ldr     w8, [x19]
     e6c:       350004e8        cbnz    w8, f08 <vgPlain_invoke_gdbserver+0xc0>

That's not quite the same, the two words after the addert are different.
Comment 10 Paul Floyd 2025-10-04 07:55:05 UTC
I should have done an objdump of none-arm64-freebsd
When I do that the words are the same.
Comment 11 Paul Floyd 2025-10-04 17:39:08 UTC
Calling getregs before gives me c0000998 in x0.
Putting 0x8BADFOOD also in x[1]  shows up correctly if I add a 2nd argument.

Really scratching my head with this.
Comment 12 Paul Floyd 2025-10-06 08:22:00 UTC
Setting

 debug.ptrace_attach_transparent=0

(needs root privs)

works around the arm64 gdbserver failures.
Comment 13 Paul Floyd 2025-10-06 20:15:05 UTC
For the arm64 gdbserver issues

sudo sysctl debug.ptrace_attach_transparent=0

is a workaround

See also

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=290008

So now on arm64 I have the following issues

memcheck/tests/freebsd/pdfork_pdkill     (stderr)
^^^
One extra Conditional jump or move depends on uninitialised value(s)

memcheck/tests/freebsd/timing_safe       (stderr)
^^^
Should be fixed in FreeBSD soon

memcheck/tests/thread_alloca             (stderr)
^^^
has been slightly flaky for a long time, now consistently flaky

+Syscall param sigprocmask(set) points to uninitialised byte(s)
Comment 14 Paul Floyd 2025-10-06 20:16:19 UTC
FreeBSD 15.0-ALPHA5 aarch64 contains a fix for the timingsafe_memcmp