Bundled GCC on windows doesn't pick up system link paths

[alexcrichton]

A system mingw-w64 gcc installation has many many libs in /mingw64/$triple/lib which gcc will make available for linking by default. The bundled GCC that rust provides, however, does not link against these libraries by default (they are not in any -L flag). This means that gcc invocations which would otherwise work on the system begin to fail because the bundled GCC is preferred.

For a concrete example, see rust-lang/cargo#620.

cc @vadimcn

[alexcrichton]

This has also broken the cargo nightlies and builds

error: linking with `gcc` failed: exit code: 1
note: gcc '-m64' '-L' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib' '-o' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\cargo.exe' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\cargo.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' '-fno-lto' '-fno-use-linker-plugin' '-Wl,--gc-sections' '-shared-libgcc' '-Wl,--enable-long-section-names' '-Wl,--nxcompat' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libnative-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\libcargo-3045fb9d724e9092.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libglob-54e4fdcda4a7c968.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libflate2-7571a86064e76c6d.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libterm-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libtar-6ccba66e5440ac37.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libtoml-a3516e82ea71aded.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libgit2-09174fe7d724f48b.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libtime-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libsemver-588b3e8c2b1d977b.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libdocopt-c165ee05d1fe3dd7.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\liblibgit2-fcba23a36b59f31c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libopenssl-static-sys-6ae299ea46a69e57.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libdebug-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libcurl-f3f9ef32955b72e6.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\liburl-921578b148f50e06.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libserialize-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\liblog-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libregex-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libencoding-4805bc5305f7cd87.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libstd-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libsync-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\librand-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\librustrt-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libcollections-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\liballoc-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\liblibc-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libunicode-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libcore-4e7c5e5c.rlib' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\flate2-7571a86064e76c6d' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\curl-sys-9d9d6a6df3c98f14' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\libgit2-fcba23a36b59f31c' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\openssl-static-sys-6ae299ea46a69e57' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\libssh2-static-sys-c29829f2cf163ffd' '-L' 'C:\bot\slave\cargo-win64-64\build\.rust' '-L' 'C:\bot\slave\cargo-win64-64\build' '-Wl,--whole-archive' '-Wl,-Bstatic' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-lws2_32' '-lkernel32' '-lwinhttp' '-lrpcrt4' '-lole32' '-lws2_32' '-lbcrypt' '-lcrypt32' '-lws2_32' '-lgcc_s' '-lcompiler-rt'
note: ld: cannot find -lwinhttp
ld: cannot find -lrpcrt4
ld: cannot find -lole32
ld: cannot find -lbcrypt

[brson]

I'm not sure this is a bug if cargo is relying on random mingw libraries - eventually we will switch away from mingw completely and those libs will not be available.

[vadimcn]

How did cargo machine end up with a bundled gcc? Does it use windows installer?!

[alexcrichton]

The cargo builders do not have a pre-installed rustc and they are run in a mingw-w64 shell (and mingw32 for now). The first step of a cargo build on the bots is to download the latest rust nightly installer and unpack it.

[vadimcn]

Hmm, I'd have expected them to use a snapshot compiler...

[jhasse]

@brson What are you planning to use instead of MinGW?

[retep998]

If we want to link Rust programs against Windows libraries we'll have to either bundle those libraries, depend on the system MinGW's copy of them, or depend on the MSVC toolchain (not likely to happen). So unless Cargo dependencies are going to start bundling the system libraries, I don't see how we can really avoid a dependency on a system MinGW for any non-trivial project.

[vadimcn]

Well, we could:

1. Tell users that if a program uses anything other than standard Rust libraries, they need to install a full mingw toolchain. This is the current status-quo.
2. Bundle more (all?) of mingw-provided import libraries. This bloats the download package though, which is why I didn't do this.
3. Tell users to use direct linking to dlls (see here). Unfortunately, this option is not very ergonomic at the moment: adding %SystemRoot%\System32 to the library search path does not work because ld chokes on kernel32.dll. However if the relevant system dlls are copied out to a separate folder, direct linking seems to work fine.
   3a. Figure out how to make gcc find bundled import libraries before searching System32.
4. Tell users to use dlltool utility (which we bundle) to create import libraries themselves.
   4a. Teach rustc or cargo to do it for them.

[brson]

Nominating. From what I understand Cargo's mingw dependencies are preventing us from accomplishing #11782.

[brson]

@jhasse We rely on mingw principally for the GNU linker, and would prefer to instead use the msvc linker.