Bug 444488

In the discussion on valgrind-users mailing list,
Paul reported tthat:
  'It looks like "stack_cache_actsize" in libc moved to be _dl_stack_cache_actsize in ld-linux-x86-64.so.2'

Is there a way to modify the glibc glibc.pthread.stack_cache_size tunable from valgrind ?
Or do we assume that the user has to tune this ?
Or do we do an alternate implementation of the current valgrind hack using _dl_stack_cache_actsize in ld-linux-x86-64.so.2 ?

(In reply to Philippe Waroquiers from comment #1)
> In the discussion on valgrind-users mailing list,
> Paul reported tthat:
>   'It looks like "stack_cache_actsize" in libc moved to be
> _dl_stack_cache_actsize in ld-linux-x86-64.so.2'
> 
> Is there a way to modify the glibc glibc.pthread.stack_cache_size tunable
> from valgrind ?

tunables are set by the GLIBC_TUNABLES environment variable
https://www.gnu.org/software/libc/manual/html_node/Tunables.html

We can set/add to that GLIBC_TUNABLES environment variable in coregrind/m_initimg/initimg-linux.c setup_client_env () where we also set the LD_PRELOAD environment variable.

(In reply to Mark Wielaard from comment #2)
> (In reply to Philippe Waroquiers from comment #1)
> > In the discussion on valgrind-users mailing list,
> > Paul reported tthat:
> >   'It looks like "stack_cache_actsize" in libc moved to be
> > _dl_stack_cache_actsize in ld-linux-x86-64.so.2'
> > 
> > Is there a way to modify the glibc glibc.pthread.stack_cache_size tunable
> > from valgrind ?
> 
> tunables are set by the GLIBC_TUNABLES environment variable
> https://www.gnu.org/software/libc/manual/html_node/Tunables.html
> 
> We can set/add to that GLIBC_TUNABLES environment variable in
> coregrind/m_initimg/initimg-linux.c setup_client_env () where we also set
> the LD_PRELOAD environment variable.

When running with a newer glibc, we should also avoid producing an error message 
that the old way to disable the stack cache is not working.
Likely this implies to detect the version of glibc.

Thanks for adding me to the CC Philippe.

If I do this:
export GLIBC_TUNABLES="glibc.pthread.stack_cache_size=0"

Then helgrind/tests/tls_threads fails with just
+--21937:0:   sched WARNING: pthread stack cache cannot be disabled!

Without the env var there are a load of 

+Possible data race during write of size 8 at 0x........ by thread #x

errors

Do we have a way of knowing that GLIBC_TUNABLES did something so that we don't need to twiddle with stack_cache_actsize?
Also --sim-hints=no-nptl-pthread-stackcache isn't turned on by default. Do we want to check for it in setup_client_env ()  and only put GLIBC_TUNABLES in the environment if it is used? Or perhaps add a new simhint.

Bonus points for handling GLIBC_TUNABLES already set by the tuner, and add or replace glibc.pthread.stack_cache_size

This doesn't seem to help the example being discussed in the valgrind-users list.

(In reply to Paul Floyd from comment #4)
> Thanks for adding me to the CC Philippe.
> 
> If I do this:
> export GLIBC_TUNABLES="glibc.pthread.stack_cache_size=0"
> 
> Then helgrind/tests/tls_threads fails with just
> +--21937:0:   sched WARNING: pthread stack cache cannot be disabled!
> 
> Without the env var there are a load of 
> 
> +Possible data race during write of size 8 at 0x........ by thread #x
> 
> errors
> 
> Do we have a way of knowing that GLIBC_TUNABLES did something so that we
> don't need to twiddle with stack_cache_actsize?
If we can detect the glibc version, then we can avoid using the old stack_cache_actsize hack.

> Also --sim-hints=no-nptl-pthread-stackcache isn't turned on by default. Do
> we want to check for it in setup_client_env ()  and only put GLIBC_TUNABLES
> in the environment if it is used? Or perhaps add a new simhint.
I think we should check and use the existing hint. Current users of the hint 
will/should have the same behaviour whatever the glibc version.

> 
> Bonus points for handling GLIBC_TUNABLES already set by the tuner, and add
> or replace glibc.pthread.stack_cache_size
> 
> This doesn't seem to help the example being discussed in the valgrind-users
> list.

> I think we should check and use the existing hint. Current users of the hint 
> will/should have the same behaviour whatever the glibc version.

There is gnu_get_libc_version(). Now, how to call that without breaking musl.

(In reply to Paul Floyd from comment #6)
> > I think we should check and use the existing hint. Current users of the hint 
> > will/should have the same behaviour whatever the glibc version.
> 
> There is gnu_get_libc_version(). Now, how to call that without breaking musl.
I suspect it might be problematic to call a glibc function (as a host function) during client image initialisation.
So, we might have to always set the tunable env variable as part of the client initialisation,
and call gnu_get_libc_version before producing an error message that the old hack did not work.

I'm having a go at this.

Setting GLIBC_TUNABLES isn't too difficult (though I haven't yet added handling when it is already set).

glibc version seems a lot harder.

In glibc:
paulf> nm /lib64/libc.so.6  | grep libc_version
00000000000296c0 t __gnu_get_libc_version
00000000000296c0 W gnu_get_libc_version
00000000001b8ada r __libc_version

I don't see how to access any of these.

Created attachment 153718 [details]
patch to set tunable

This fixes helgrind/tests/tls_threads
But it only adds GLIBC_TUNABLES, it doesn't try to modify an existing one.

Created attachment 153723 [details]
Updated patch now handles existing GLIBC_TUNABLES and no lnger prints a warning

Looks like this is now about ready. I haven't yet tested on other platforms.

I did try to chjeckout and build the latest code on Alpine but there was a link error (without these changes)..

I've tested this on FreeBSD and musl. Both fail helgrind/tests/tls_threads both before and after so no change with this patch.

commit 2de91d914cc5ed4ed7ea8e70d5e06c46e991f39f (HEAD -> master, origin/master, origin/HEAD)
Author: Paul Floyd <pjfloyd@wanadoo.fr>
Date:   Fri Dec 23 16:49:20 2022 +0100

    Bug 444488 - Use glibc.pthread.stack_cache_size tunable
    
    Try to use GLIBC_TUNABLES to disable the pthread stack
    cache.