Bug 294047

Summary: valgrind does not correctly emulate prlimit64(..., RLIMIT_NOFILE, ...)
Product: [Developer tools] valgrind Reporter: Matthias Schwarzott <zzam>
Component: generalAssignee: Julian Seward <jseward>
Status: RESOLVED FIXED    
Severity: normal CC: austinenglish, kde-bugs, tom
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Unlisted Binaries   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: valgrind-improve-prlimit64-v5.patch

Description Matthias Schwarzott 2012-02-14 07:50:29 UTC 
Version:           unspecified
OS:                Linux

The prlimit64 syscall is not correctly emulated by valgrind.
On 32bit platforms setrlimit64 and getrlimit64 are implemented using prlimit64.


Reproducible: Always

Steps to Reproduce:
Lowering RLIMIT_NOFILE using prlimit64.


Actual Results:  
valgrind can no longer open private files, as fcntl(FD_DUPFD, ...) will fail.


Expected Results:  
prlimit64 should work equivalent to setrlimit and getrlimit.
Comment 1 Matthias Schwarzott 2012-02-18 10:13:01 UTC 
Created attachment 68897 [details]
valgrind-improve-prlimit64-v5.patch

The attached patch does this:
Change prlimit64 to behave like a combination of the existing setrlimit and getrlimit.
Run emulation only if ARG0 (pid) is 0 or equal getpid()
improve setrlimit (like new prlimit64) to return EINVAL if softlimit>hardlimit
Add tests (using getrlimit64 and setrlimit64)

TODO: Tests using prlimit and prlimit64 (perhaps with different pid
values)

on x86:
getrlimit -> syscall getrlimit
setrlimit -> syscall setrlimit
prlimit -> syscall prlimit64
getrlimit64 -> syscall prlimit64
setrlimit64 -> syscall prlimit64
prlimit64 -> syscall prlimit64

on amd64:
getrlimit -> syscall getrlimit
setrlimit -> syscall setrlimit
prlimit -> syscall prlimit64
getrlimit64 -> syscall getrlimit
setrlimit64 -> syscall setrlimit
prlimit64 -> syscall prlimit64
Comment 2 Tom Hughes 2012-03-01 13:47:00 UTC 
Committed as r12411.
Comment 3 Philippe Waroquiers 2012-09-24 20:52:12 UTC 
*** Bug 275673 has been marked as a duplicate of this bug. ***