Created attachment 88736 [details] C source testcase for 3.10.0 hugepage alignment bug I chose 3.9.0 because that is the most recent in the list, but the failure is actually with the just released 3.10.0. Valgrind exits with an error when you try to mmap() a 2MB or 1GB hugepage. It works fine if you request a normal 4K page. The problem is that even when NULL is specified as the address in the mmap() call, Valgrind substitutes a specific address and adds the MAP_FIXED flag. But this address Valgrind chooses is neither 2MB nor 1GB aligned, and thus the allocation fails. This bug may be complicated to reproduce unless you already have hugepages configured on your system. But the cause, confirmation, and the fix should be clear from the following transcript showing the results with the attached test case both with and without Valgrind. nate@haswell:~$ gcc -O2 -Wall -Wextra valgrind_hugepage_mmap_bug.c -o valgrind_hugepage_mmap_bug nate@haswell:~/bitvector$ valgrind_hugepage_mmap_bug 4KB Opening /dev/zero Trying mmap() for 4KB pages Writing 100000 bytes. Success nate@haswell:~/bitvector$ valgrind_hugepage_mmap_bug 2MB Opening /var/lib/hugetlbfs/global/pagesize-2097152/page Trying mmap() for 2MB pages Writing 100000 bytes. Success nate@haswell:~/bitvector$ valgrind_hugepage_mmap_bug 1GB Opening /var/lib/hugetlbfs/global/pagesize-1073741824/page Trying mmap() for 1GB pages Writing 100000 bytes. Success nate@haswell:~/bitvector$ valgrind --version valgrind-3.10.0 nate@haswell:~/bitvector$ valgrind valgrind_hugepage_mmap_bug 4KB ==30701== Memcheck, a memory error detector ==30701== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==30701== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==30701== Command: valgrind_hugepage_mmap_bug 4KB ==30701== Opening /dev/zero Trying mmap() for 4KB pages Writing 100000 bytes. Success ==30701== ==30701== HEAP SUMMARY: ==30701== in use at exit: 0 bytes in 0 blocks ==30701== total heap usage: 0 allocs, 0 frees, 0 bytes allocated ==30701== ==30701== All heap blocks were freed -- no leaks are possible ==30701== ==30701== For counts of detected and suppressed errors, rerun with: -v ==30701== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) nate@haswell:~/bitvector$ valgrind valgrind_hugepage_mmap_bug 2MB ==30703== Memcheck, a memory error detector ==30703== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==30703== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==30703== Command: valgrind_hugepage_mmap_bug 2MB ==30703== Opening /var/lib/hugetlbfs/global/pagesize-2097152/page Trying mmap() for 2MB pages mmap: Invalid argument ==30703== ==30703== HEAP SUMMARY: ==30703== in use at exit: 0 bytes in 0 blocks ==30703== total heap usage: 1 allocs, 1 frees, 568 bytes allocated ==30703== ==30703== All heap blocks were freed -- no leaks are possible ==30703== ==30703== For counts of detected and suppressed errors, rerun with: -v ==30703== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) nate@haswell:~/bitvector$ valgrind valgrind_hugepage_mmap_bug 1GB ==30707== Memcheck, a memory error detector ==30707== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==30707== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==30707== Command: valgrind_hugepage_mmap_bug 1GB ==30707== Opening /var/lib/hugetlbfs/global/pagesize-1073741824/page Trying mmap() for 1GB pages mmap: Invalid argument ==30707== ==30707== HEAP SUMMARY: ==30707== in use at exit: 0 bytes in 0 blocks ==30707== total heap usage: 1 allocs, 1 frees, 568 bytes allocated ==30707== ==30707== All heap blocks were freed -- no leaks are possible ==30707== ==30707== For counts of detected and suppressed errors, rerun with: -v ==30707== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) nate@haswell:~/bitvector$ strace valgrind_hugepage_mmap_bug 4KB ...open("/dev/zero", O_RDWR|O_CREAT, 0755) = 3 ...mmap(NULL, 100000, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0x7f8182d45000 nate@haswell:~/bitvector$ strace valgrind_hugepage_mmap_bug 2MB ...open("/var/lib/hugetlbfs/global/pagesize-2097152/page", O_RDWR|O_CREAT, 0755) = 3 ...mmap(NULL, 100000, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0x2aaaaac00000 nate@haswell:~/bitvector$ strace valgrind_hugepage_mmap_bug 1GB ...open("/var/lib/hugetlbfs/global/pagesize-1073741824/page", O_RDWR|O_CREAT, 0755) = 3 ...mmap(NULL, 100000, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0x2aaac0000000 nate@haswell:~/bitvector$ strace valgrind valgrind_hugepage_mmap_bug 4KB ...open("/dev/zero", O_RDWR|O_CREAT, 0755) = 3 ...mmap(0x4026000, 100000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0x4026000 nate@haswell:~/bitvector$ strace valgrind valgrind_hugepage_mmap_bug 2MB ...open("/var/lib/hugetlbfs/global/pagesize-2097152/page", O_RDWR|O_CREAT, 0755) = 3 ...mmap(0x4026000, 100000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument) nate@haswell:~/bitvector$ strace valgrind valgrind_hugepage_mmap_bug 1GB ...open("/var/lib/hugetlbfs/global/pagesize-1073741824/page", O_RDWR|O_CREAT, 0755) = 3 ...mmap(0x4026000, 100000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)
*** This bug has been marked as a duplicate of bug 333051 ***
Hi Tom -- I agree on the dup. I entered this one separately because the title on the original wasn't very clear about the problem, and because it didn't make clear that this remains a problem with the latest version. If you think it appropriate, could you update the title and version on that one, and add this one as a See Also? Thanks! --nate On Thu, Sep 18, 2014 at 1:36 AM, Tom Hughes <tom@compton.nu> wrote: > https://bugs.kde.org/show_bug.cgi?id=339163 > > Tom Hughes <tom@compton.nu> changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC| |tom@compton.nu > Resolution|--- |DUPLICATE > Status|UNCONFIRMED |RESOLVED > > --- Comment #1 from Tom Hughes <tom@compton.nu> --- > > > *** This bug has been marked as a duplicate of bug 333051 *** > > -- > You are receiving this mail because: > You reported the bug.
Done, though I'm not at all sure what we can do in this case as I suspect there is no easy way for us to know that we need huge page alignment in this case. If you were using MAP_HUGETLB then it would be easy to detect, but when you're doing an ordinary map from a file that is on a hugetlbfs filesystem it may be very hard to detect.