| Message ID | 201610041224.00059.cehoyos@ag.or.at |
|---|---|
| State | Accepted |
| Commit | a2c5f5aacf9d1e53602864cde21ab3c8b730a617 |
| Headers | show |
On Tue, Oct 04, 2016 at 12:24:00PM +0200, Carl Eugen Hoyos wrote: > Hi! > > Sorry if I miss something but with this patch, the hardening_check > script succeeds here both for x86_32 and x86_64 (static and shared). > > Please comment, Carl Eugen only case i found that breaks is with --enable-libxavs /usr/bin/ld: /usr/local/lib/libxavs.a(common.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /usr/local/lib/libxavs.a: could not read symbols: Bad value not sure our configure should detect this or if thats beyond its scope [...]
2016-10-04 18:00 GMT+02:00 Michael Niedermayer <michael@niedermayer.cc>: > On Tue, Oct 04, 2016 at 12:24:00PM +0200, Carl Eugen Hoyos wrote: >> >> Sorry if I miss something but with this patch, the hardening_check >> script succeeds here both for x86_32 and x86_64 (static and shared). > only case i found that breaks is with --enable-libxavs > /usr/bin/ld: /usr/local/lib/libxavs.a(common.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC > /usr/local/lib/libxavs.a: could not read symbols: Bad value Patch sent that fixes this issue. I would have expected that this (pie) patch does not work on x86_32 but the binary runs fine here: Am I missing something or should I apply to get this tested? Carl Eugen
2016-10-05 15:14 GMT+02:00 Carl Eugen Hoyos <ceffmpeg@gmail.com>:
> Patch sent that fixes this issue.
I'll apply both patches if there are no objections.
Carl Eugen
On 04.10.2016 12:24, Carl Eugen Hoyos wrote: > Sorry if I miss something but with this patch, the hardening_check > script succeeds here both for x86_32 and x86_64 (static and shared). This script uses a very simplistic approach for testing position independent executables. I think it just does the equivalent of 'readelf -h $PROGRAM | grep Type'. If the Type is EXEC, it's a normal executable, and if it is DYN, it assumes it's compiled as PIE. However, that doesn't guarantee that the executable is actually position independent, i.e. does not contain text relocations. > --- a/configure > +++ b/configure > @@ -3577,6 +3577,8 @@ case "$toolchain" in > add_cppflags -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 > add_cflags -fno-strict-overflow -fstack-protector-all > add_ldflags -Wl,-z,relro -Wl,-z,now > + add_cflags -fPIE I think this should be -fPIC, at least when building shared libraries. That's how I understand the gcc manual [1]: -fpie -fPIE These options are similar to -fpic and -fPIC, but generated position independent code can be only linked into executables. > + add_ldexeflags -fPIE -pie > ;; > ?*) > die "Unknown toolchain $toolchain" > -- 1.7.10.4 In general, enabling PIE for toolchain=hardened is a good idea. But According to [2] PIE doesn't work on hppa and m68k, so it shouldn't get enabled for these architectures. On 05.10.2016 15:14, Carl Eugen Hoyos wrote: > I would have expected that this (pie) patch does not work on x86_32 > but the binary runs fine here: Am I missing something or should I > apply to get this tested? The problem on x86_32 is that libavcodec, libavutil, etc. use text relocations in hand-written assembler code, so these libraries won't be position independent, unless using --disable-asm. Now, when producing shared libraries, the ffmpeg binary is actually position independent, just not libavcodec, libavutil... However, when linking statically, the ffmpeg binary contains the text relocations from the hand-written assembler code and is thus not really position independent. This can be tested e.g. with scanelf from pax-utils [3]. * shared PIE build on x86_32 (no text relocations): $ scanelf -t ./ffmpeg TYPE TEXTREL FILE ET_DYN - ./ffmpeg * static PIE build on x86_32 (with text relocations): $ scanelf -t ./ffmpeg TYPE TEXTREL FILE ET_DYN TEXTREL ./ffmpeg The '-T' options shows were exactly the text relocations are. Best regards, Andreas 1: https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html 2: https://wiki.debian.org/Hardening#DEB_BUILD_HARDENING_PIE_.28gcc.2Fg.2B-.2B-_-fPIE_-pie.29 3: https://wiki.gentoo.org/wiki/Hardened/PaX_Utilities
2016-10-12 19:04 GMT+02:00 Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>: > On 04.10.2016 12:24, Carl Eugen Hoyos wrote: >> Sorry if I miss something but with this patch, the hardening_check >> script succeeds here both for x86_32 and x86_64 (static and shared). > > This script uses a very simplistic approach for testing position > independent executables. > I think it just does the equivalent of 'readelf -h $PROGRAM | grep Type'. > If the Type is EXEC, it's a normal executable, and if it is DYN, it > assumes it's compiled as PIE. > However, that doesn't guarantee that the executable is actually position > independent, i.e. does not contain text relocations. My understanding of PIE is (very) limited but I suspect text relocations and PIE do not exclude each other. >> --- a/configure >> +++ b/configure >> @@ -3577,6 +3577,8 @@ case "$toolchain" in >> add_cppflags -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 >> add_cflags -fno-strict-overflow -fstack-protector-all >> add_ldflags -Wl,-z,relro -Wl,-z,now >> + add_cflags -fPIE > > I think this should be -fPIC, at least when building shared libraries. Afaiu, shared libraries and PIE do exclude each other, pic is already supported in FFmpeg (--enable-pic) but this patch is only meant for PIE. > That's how I understand the gcc manual [1]: > -fpie > -fPIE > These options are similar to -fpic and -fPIC, but generated position > independent code can be only linked into executables. So shared libraries and PIE exclude each other but PIE is for static linking what pic is for dynamic linking. Good to know! >> + add_ldexeflags -fPIE -pie >> ;; >> ?*) >> die "Unknown toolchain $toolchain" >> -- 1.7.10.4 > > In general, enabling PIE for toolchain=hardened is a good idea. > But According to [2] PIE doesn't work on hppa and m68k, so it > shouldn't get enabled for these architectures. I was convinced that my ancient Linux system wouldn't know about ASLR but to my surprise, I was able to reproduce that my patch actually works (56bit entropy iiuc). My hppa test-system is currently down (and all reports about pie not working on hppa seem to be from 2008), I will try to test, in any case, I will commit the patch soon. I will not be able to test on m68k. Carl Eugen
On 12.10.2016 23:44, Carl Eugen Hoyos wrote: > 2016-10-12 19:04 GMT+02:00 Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>: >> On 04.10.2016 12:24, Carl Eugen Hoyos wrote: >>> Sorry if I miss something but with this patch, the hardening_check >>> script succeeds here both for x86_32 and x86_64 (static and shared). >> >> This script uses a very simplistic approach for testing position >> independent executables. >> I think it just does the equivalent of 'readelf -h $PROGRAM | grep Type'. >> If the Type is EXEC, it's a normal executable, and if it is DYN, it >> assumes it's compiled as PIE. > >> However, that doesn't guarantee that the executable is actually position >> independent, i.e. does not contain text relocations. > > My understanding of PIE is (very) limited but I suspect text relocations > and PIE do not exclude each other. As I understand it the literal meaning of position independent code is code without text relocations, because those are what makes the code position dependent. So in this literal sense PIE and text relocations exclude each other. (The -fPIC/-fPIE flags tell the compiler to produce code without text relocations.) The additional complication for executables is that ASLR doesn't work for normal executables, because they are always mapped to the same point in address space. The basic idea of PIE is to build the executables actually as shared libraries, so that they can benefit from ASLR just as normal shared libraries. (The '-pie' flag tells the linker to produce such an executable.) However, such a '-pie' executable is not necessarily position independent in the literal sense, i.e. might contain text relocations. In that sense PIE and text relocations don't exclude each other. If you want both NX and ASLR security features for an executable it has to be built with '-pie' and must not contain text relocations. >>> --- a/configure >>> +++ b/configure >>> @@ -3577,6 +3577,8 @@ case "$toolchain" in >>> add_cppflags -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 >>> add_cflags -fno-strict-overflow -fstack-protector-all >>> add_ldflags -Wl,-z,relro -Wl,-z,now >>> + add_cflags -fPIE >> >> I think this should be -fPIC, at least when building shared libraries. > > Afaiu, shared libraries and PIE do exclude each other, pic is already > supported in FFmpeg (--enable-pic) but this patch is only meant for > PIE. > >> That's how I understand the gcc manual [1]: >> -fpie >> -fPIE >> These options are similar to -fpic and -fPIC, but generated position >> independent code can be only linked into executables. > > So shared libraries and PIE exclude each other but PIE is for > static linking what pic is for dynamic linking. > Good to know! I think you misunderstood this. The problem with -fPIE is that it allows the compiler to make optimizations only suitable if the resulting object ends up in an executable. However, it is entirely possible to build the objects with -fPIC and produce an executable with '-pie'. These same objects can also be used to build normal shared libraries, while objects compiled with '-fPIE' shouldn't be used for that. And '-pie' is also useful for dynamic linking. So using -fPIC in CFLAGS and '-fPIC -pie' in LDEXEFLAGS should always work. >>> + add_ldexeflags -fPIE -pie >>> ;; >>> ?*) >>> die "Unknown toolchain $toolchain" >>> -- 1.7.10.4 >> >> In general, enabling PIE for toolchain=hardened is a good idea. > >> But According to [2] PIE doesn't work on hppa and m68k, so it >> shouldn't get enabled for these architectures. > > I was convinced that my ancient Linux system wouldn't know > about ASLR but to my surprise, I was able to reproduce that > my patch actually works (56bit entropy iiuc). > My hppa test-system is currently down (and all reports about > pie not working on hppa seem to be from 2008), The information from the wiki might be outdated... > I will try to test, > in any case, I will commit the patch soon. > > I will not be able to test on m68k. I can't test hppa or m68k directly either. Best regards, Andreas
2016-10-04 12:24 GMT+02:00 Carl Eugen Hoyos <cehoyos@ag.or.at>: > Sorry if I miss something but with this patch, the hardening_check > script succeeds here both for x86_32 and x86_64 (static and shared). Tested successfully on x86_64 and x86_32 Linux (pie actually works on my very old system). On Debian hppa, the patch makes no difference, I cannot test m68k. Since my gcc manual states that -fPIE is needed for -pie, I decided to use that. Patch applied, Carl Eugen
On 14.10.2016 15:02, Carl Eugen Hoyos wrote: > 2016-10-04 12:24 GMT+02:00 Carl Eugen Hoyos <cehoyos@ag.or.at>: > >> Sorry if I miss something but with this patch, the hardening_check >> script succeeds here both for x86_32 and x86_64 (static and shared). > > Tested successfully on x86_64 and x86_32 Linux (pie actually works > on my very old system). On Debian hppa, the patch makes no > difference, I cannot test m68k. I guess gcc nowadays just ignores the flag if it wouldn't work. > Since my gcc manual states that -fPIE is needed for -pie, I decided > to use that. In practice it shouldn't make a difference, because -fPIC is added to CFLAGS after -fPIE anyway, which I think overrides it. Best regards, Andreas
On Thu, Oct 13, 2016 at 12:56:56AM +0200, Andreas Cadhalpun wrote: > On 12.10.2016 23:44, Carl Eugen Hoyos wrote: > > 2016-10-12 19:04 GMT+02:00 Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>: > >> On 04.10.2016 12:24, Carl Eugen Hoyos wrote: > >>> Sorry if I miss something but with this patch, the hardening_check > >>> script succeeds here both for x86_32 and x86_64 (static and shared). > >> > >> This script uses a very simplistic approach for testing position > >> independent executables. > >> I think it just does the equivalent of 'readelf -h $PROGRAM | grep Type'. > >> If the Type is EXEC, it's a normal executable, and if it is DYN, it > >> assumes it's compiled as PIE. > > > >> However, that doesn't guarantee that the executable is actually position > >> independent, i.e. does not contain text relocations. > > > > My understanding of PIE is (very) limited but I suspect text relocations > > and PIE do not exclude each other. > > As I understand it the literal meaning of position independent code is > code without text relocations, because those are what makes the code > position dependent. So in this literal sense PIE and text relocations > exclude each other. > (The -fPIC/-fPIE flags tell the compiler to produce code without text > relocations.) > > The additional complication for executables is that ASLR doesn't work for > normal executables, because they are always mapped to the same point in > address space. The basic idea of PIE is to build the executables actually > as shared libraries, so that they can benefit from ASLR just as normal > shared libraries. > (The '-pie' flag tells the linker to produce such an executable.) > > However, such a '-pie' executable is not necessarily position independent > in the literal sense, i.e. might contain text relocations. > In that sense PIE and text relocations don't exclude each other. > > If you want both NX and ASLR security features for an executable it has > to be built with '-pie' and must not contain text relocations. this should not be true the difference between text relocations and lack there off is that without text relocations a binary is loaded and written into memory with text relocations the binary is loaded the addresses for relocations updated and writen into memory. There is at a theoretical level no difference in required access rights write to memory is neccessary at the load stage, no execute is needed here and once done rights can be fliped over into execute without write This may very well not work out that way in gnu linux but thats a implementation problem then not a fundamental issue in NX+ASLR+TEXRELs That is unless iam missing something also a simple test: gcc xtest.c -pie -m32 -o xtest int main() { void *ref; asm ( "mov $main, %0" :"=r"(ref) ); printf("? %p\n", ref); //can we read it ? printf("R %d\n", *(int*)ref); //can we write it ? *(int*)ref = 123; return 0; } Executing this shows that the write is prevented and segfaults, the address is different on each run and we have a text relocation in it thats on a ancient ubuntu without special security patches that i remember [...]
On Fri, Oct 14, 2016 at 06:28:32PM +0200, Michael Niedermayer wrote: > On Thu, Oct 13, 2016 at 12:56:56AM +0200, Andreas Cadhalpun wrote: > > On 12.10.2016 23:44, Carl Eugen Hoyos wrote: > > > 2016-10-12 19:04 GMT+02:00 Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>: > > >> On 04.10.2016 12:24, Carl Eugen Hoyos wrote: > > >>> Sorry if I miss something but with this patch, the hardening_check > > >>> script succeeds here both for x86_32 and x86_64 (static and shared). > > >> > > >> This script uses a very simplistic approach for testing position > > >> independent executables. > > >> I think it just does the equivalent of 'readelf -h $PROGRAM | grep Type'. > > >> If the Type is EXEC, it's a normal executable, and if it is DYN, it > > >> assumes it's compiled as PIE. > > > > > >> However, that doesn't guarantee that the executable is actually position > > >> independent, i.e. does not contain text relocations. > > > > > > My understanding of PIE is (very) limited but I suspect text relocations > > > and PIE do not exclude each other. > > > > As I understand it the literal meaning of position independent code is > > code without text relocations, because those are what makes the code > > position dependent. So in this literal sense PIE and text relocations > > exclude each other. > > (The -fPIC/-fPIE flags tell the compiler to produce code without text > > relocations.) > > > > The additional complication for executables is that ASLR doesn't work for > > normal executables, because they are always mapped to the same point in > > address space. The basic idea of PIE is to build the executables actually > > as shared libraries, so that they can benefit from ASLR just as normal > > shared libraries. > > (The '-pie' flag tells the linker to produce such an executable.) > > > > However, such a '-pie' executable is not necessarily position independent > > in the literal sense, i.e. might contain text relocations. > > In that sense PIE and text relocations don't exclude each other. > > > > > If you want both NX and ASLR security features for an executable it has > > to be built with '-pie' and must not contain text relocations. > > this should not be true > the difference between text relocations and lack there off is that > without text relocations a binary is loaded and written into memory > with text relocations the binary is loaded the addresses for > relocations updated and writen into memory. > There is at a theoretical level no difference in required access rights > write to memory is neccessary at the load stage, no execute is needed > here and once done rights can be fliped over into execute without write > This may very well not work out that way in gnu linux but thats a > implementation problem then not a fundamental issue in NX+ASLR+TEXRELs > That is unless iam missing something > > also a simple test: > gcc xtest.c -pie -m32 -o xtest > int main() { > void *ref; > asm ( > "mov $main, %0" > :"=r"(ref) > ); > printf("? %p\n", ref); > //can we read it ? > printf("R %d\n", *(int*)ref); > //can we write it ? > *(int*)ref = 123; > > return 0; > } > > Executing this shows that the write is prevented and segfaults, the > address is different on each run and we have a text relocation in it > thats on a ancient ubuntu without special security patches that i > remember Heres an example for 64bit with ASLR+TEXREL+non writable code gcc xtest.c -fPIE -pie -o xtest int main() { void *ref; asm ( "movabs $main, %0" :"=r"(ref) ); printf("? %p\n", ref); //can we read it ? printf("R %d\n", *(int*)ref); //can we write it ? *(int*)ref = 123; printf("W\n"); return 0; } [...]
On 14.10.2016 18:28, Michael Niedermayer wrote: > On Thu, Oct 13, 2016 at 12:56:56AM +0200, Andreas Cadhalpun wrote: >> If you want both NX and ASLR security features for an executable it has >> to be built with '-pie' and must not contain text relocations. > > this should not be true > the difference between text relocations and lack there off is that > without text relocations a binary is loaded and written into memory > with text relocations the binary is loaded the addresses for > relocations updated and writen into memory. > There is at a theoretical level no difference in required access rights > write to memory is neccessary at the load stage, no execute is needed > here and once done rights can be fliped over into execute without write > This may very well not work out that way in gnu linux but thats a > implementation problem then not a fundamental issue in NX+ASLR+TEXRELs > That is unless iam missing something > > also a simple test: > gcc xtest.c -pie -m32 -o xtest > int main() { > void *ref; > asm ( > "mov $main, %0" > :"=r"(ref) > ); > printf("? %p\n", ref); > //can we read it ? > printf("R %d\n", *(int*)ref); > //can we write it ? > *(int*)ref = 123; > > return 0; > } > > Executing this shows that the write is prevented and segfaults, the > address is different on each run and we have a text relocation in it > thats on a ancient ubuntu without special security patches that i > remember Interesting... I was just rephrasing what I found on the web [1]: "For NX to be useful, you need to make sure that all the executable memory pages are loaded and set in stone right away; this makes text relocation impossible" Best regards, Andreas 1: https://blog.flameeyes.eu/2009/11/the-pie-is-not-exactly-a-lie/
diff --git a/configure b/configure index f593191..858f2a6 100755 --- a/configure +++ b/configure @@ -3577,6 +3577,8 @@ case "$toolchain" in add_cppflags -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 add_cflags -fno-strict-overflow -fstack-protector-all add_ldflags -Wl,-z,relro -Wl,-z,now + add_cflags -fPIE + add_ldexeflags -fPIE -pie ;; ?*) die "Unknown toolchain $toolchain"
Hi! Sorry if I miss something but with this patch, the hardening_check script succeeds here both for x86_32 and x86_64 (static and shared). Please comment, Carl Eugen From 3c5df95a022e9148f753dd2a850570080740c602 Mon Sep 17 00:00:00 2001 From: Carl Eugen Hoyos <cehoyos@ag.or.at> Date: Tue, 4 Oct 2016 12:21:41 +0200 Subject: [PATCH] configure: Enable pie for toolchain=hardened. --- configure | 2 ++ 1 file changed, 2 insertions(+)