ld.bfd 2.25.51 packaged with NDK r11 produces unsupported R_ARM_COPY relocation

[joakim-noah]

I've been using ld.bfd 2.24.90 from NDK r10e without this issue, but after updating to r11 the new binutils consistently uses the R_ARM_COPY relocation for __sF. I'm building the object file that refers to __sF from D, using the llvm-based ldc compiler. However, the exact same object file is linked without R_ARM_COPY by both the older ld.bfd 2.24.90 or a newer 2.26.20160125 that I have locally installed, so this is probably a regression in ld.bfd 2.25.51. I don't have this issue with ld.gold in NDK r11, but I'm stuck with ld.bfd because I need a certain ELF section ordering.

For reference, here are the relocations in the stdio.o object file that refers to __sF:

Relocation section '.rel.data.stdin' at offset 0x800 contains 1 entries:
 Offset     Info    Type            Sym.Value  Sym. Name
00000000  00004602 R_ARM_ABS32       00000000   __sF
Relocation section '.rel.data.stdout' at offset 0x808 contains 1 entries:
Offset     Info    Type            Sym.Value  Sym. Name
00000000  00004602 R_ARM_ABS32       00000000   __sF
Relocation section '.rel.data.stderr' at offset 0x810 contains 1 entries:
Offset     Info    Type            Sym.Value  Sym. Name
00000000  00004602 R_ARM_ABS32       00000000   __sF

This produces the following relocation in the final command-line executable:

0178a1c0  0000b214 R_ARM_COPY        0178a1c0   __sF

Running that executable fails with an error about R_ARM_COPY not being supported. I'm unsure how to reproduce this using a C/C++ sample, but I'm guessing it might be a problem there too? I don't know enough about ELF relocations to say.

[dimitry-]

R_ARM_COPY implies text-relocations in the executable and it is indeed not supported for apps targeting sdk version M and later.

I am a little bit fuzzy on how object relocations turn into R_ARM_COPY, if I had to guess I would say this is because D generated a file that lead to this relocation in final elf-file.

Does your code have inline assembly? If so that might be the source of the problem and this link could help with it: https://wiki.gentoo.org/wiki/Hardened/Textrels_Guide

If not - I would try (1) checking if ld.gold does the same thing and (2) asking D compiler team about it.

[joakim-noah]

Here's the thing, the exact same object files produce an executable without that relocation when I use older or newer versions of ld.bfd. That says to me that it is a bug in the linker, nothing to do with D. As for ld.gold, I noted that it doesn't have the issue, but that I need ld.bfd because it has linker scripts that keep ELF section ordering, whereas gold just reorders them any way it likes.

[dimitry-]

This definitely sounds like a bfd bug.

[joakim-noah]

I've packaged up some test objects to try out, along with the resulting executables produced by the older ld.bfd from NDK r10e and the newer one from r11. Here are the commands used to produce test.break and test.work from the given test.o and libdruntime-ldc.a:

/opt/ndk-r11/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -Wl,-z,nocopyreloc --sysroot=/opt/ndk-r11/platforms/android-9/arch-arm -lgcc -gcc-toolchain /opt/ndk-r11/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--warn-shared-textrel -Wl,--fatal-warnings -mthumb -lc -lm test.o lib/libdruntime-ldc.a -o test.break

/opt/android-ndk-r10e/toolchains/llvm-3.6/prebuilt/linux-x86_64/bin/clang -Wl,-z,nocopyreloc --sysroot=/opt/ndk-r11/platforms/android-9/arch-arm -lgcc -gcc-toolchain /opt/ndk-r11/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--warn-shared-textrel -Wl,--fatal-warnings -mthumb -lc -lm test.o lib/libdruntime-ldc.a -o test.work

test.o is compiled from this small D file that simply calls putc on stdout:

import core.stdc.stdio;
void main() { putc('a', stdout);}

libdruntime-ldc.a is the D runtime library, which contains core.stdc.stdio that references __sF.

ld-reloc.zip

[DanAlbert]

FYI, I've asked our binutils guy to take a look at this. Sounds like he has an idea of the patches that need to be cherry-picked, so should be doable for r12.

[joakim-noah]

Thanks, looking forward to that, I'll just use the older bfd linker in r10e for now.

[shenhanc78]

Hi, joakim-noah and Dan,

First of all, for the 2 command lines you aforementioned (thanks for providing reproducible commands), actually, they are using exactly the same binutinls binary - "/opt/ndk-r11/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9/../../../../arm-linux-androideabi/bin/ld.bfd". Because of the fact both '-gcc-toolchain' point to the same location.
So it's not a regression in binutils.

Secondly, the reason for "regression" is in clang. The earlier (ndk-r10) clang, while calling ld.bfd, passes a '-pie', while the newer one 'ndk-r11' clang not. You can observe this by adding '-v' right after clang.

Thirdly, this change is quite dubious - whether or not to add '-pie' by default affects the final binary in multiple ways. @DanAlbert is this dropping '-pie' intentional?

I pasted the 2 commands below for reference
Command generates broken binary -

/opt/ndk-r11/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -Wl,-z,nocopyreloc \
    --sysroot=/opt/ndk-r11/platforms/android-9/arch-arm -lgcc \
    -gcc-toolchain /opt/ndk-r11/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 \
    -target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd \
    -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now \
    -Wl,--warn-shared-textrel -Wl,--fatal-warnings -mthumb -lc -lm test.o \
    lib/libdruntime-ldc.a -o test.break`

Command generates good binary -

/opt/android-ndk-r10e/toolchains/llvm-3.6/prebuilt/linux-x86_64/bin/clang \
    -Wl,-z,nocopyreloc --sysroot=/opt/ndk-r11/platforms/android-9/arch-arm -lgcc \
    -gcc-toolchain /opt/ndk-r11/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 \
    -target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd \
    -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now \
    -Wl,--warn-shared-textrel -Wl,--fatal-warnings -mthumb -lc -lm test.o \
    lib/libdruntime-ldc.a -o test.work

[joakim-noah]

@shenhanc78, thanks for looking into this issue. You're right: that flag is the difference and the linker used is actually the same one from r11 with both command invocations. It appears Dan took out PIE by default from clang last year, and it was added to the makefiles instead.

I extracted the flags I was using from Android API 9, by running ndk-build V=1 on the since-removed test-libstdc++ NDK sample executable, because I wanted to have as broad Android support as possible. However, that's why I missed the -fPIE -pie flags that the NDK build scripts add for any executables compiled against Android API 16 or above, when PIE began being supported.

I will use those PIE flags instead, as the market share of Android API 15 and older is small.

[DanAlbert]

First of all, for the 2 command lines you aforementioned (thanks for providing reproducible commands), actually, they are using exactly the same binutinls binary - "/opt/ndk-r11/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9/../../../../arm-linux-androideabi/bin/ld.bfd". Because of the fact both '-gcc-toolchain' point to the same location.
So it's not a regression in binutils.

@joakim-noah: can you comment on that? Was that an error when copying the information into the bug, or is that right?

@shenhanc78: Dropping -pie by default was intentional; Android did not support PIE executables until android-16 (Jelly Bean). @joakim-noah: if you're building executables for pre-JB with -pie, they won't run.

[joakim-noah]

@joakim-noah: can you comment on that? Was that an error when copying the information into the bug, or is that right?

I did comment on it above, "You're right: that flag is the difference and the linker used is actually the same one from r11 with both command invocations." I had tried to make the command-lines as identical as possible by using the same --sysroot and -gcc-toolchain from NDK 11, with only the compiler different.

I didn't know that meant they'd use the same linker too, I only checked after @shenhanc78 mentioned it and he's right. The only difference is what he said: the old clang 3.6 in NDK r10e was passing the -pie flag to the linker by default, whereas the new clang 3.8 in NDK r11 isn't. I should have checked this myself before filing the issue.

@joakim-noah: if you're building executables for pre-JB with -pie, they won't run.

Yes, I know, that's fine with me. Apparently, they only ever ran for me on 4.1 and above because clang was passing -pie by default before.

I suppose this issue shows that some other flag will now have to be passed to the linker if you want to support Android APIs before 16, ie non-PIE? You could change the topic to that if you like.