A new compiler flag: "link everything statically or die, dammit!"

[golddranks]

My adventures with static linking and native dependencies have been frustrating – because of the multitude of different build.rs scripts each configured in a different way, the build environment is fragile. Lately I made some changes to my environment, and since then, I've been trying to get all the native deps linking statically again – and some of them quite stubbornly refuse to do so.

The general problem with build.rs is hard to fix, but I think at least the process of troubleshooting the build could be made less frustrating. I'm thinking something like passing a flag to cargo or rustc, signalling that "I'm expecting the build to be fully statically linked, so if some of the libraries passes dynamic linking flags, stop building and tell me the name of that traitor".

At the moment, the build just silently accepts dynamic flags, resulting, after a long compilation, in a binary that I'm expecting to be able to run in an empty Linux userspace, but which fails. This is because rustc doesn't know my intent. Should we not be allowed to signal that intent?

[golddranks]

If such a flag existed, if could be passed to the build.rs scripts too, so they could abort and tell what's wrong if they couldn't fulfil the intent. At the moment there's many scripts that specify their own environment variables to signal static linking, but as the build.rs scripts are quite varied, their semantics vary too, from "Couldn't do that? Try to build with dynamic then" to "Sorry, no can do, here's an error." Plus there may be multiple different ways to setup the configuration (env vars, pkg-config, calling some binary and getting flags from there, trying some sensible defaults...), and they have some precedence order, which may be different in each crate. Having rustc to abort and report error in such a situation would encourage more unified semantics in the build.rs ecosystem too.

[retep998]

If some crate asks to link foo.lib, how do you know whether that library is a static library or a dynamic library? Also on some platforms cough Windows cough you can't do anything without linking to the system libraries which are all dynamic.

[golddranks]

The build.rs build scripts pass flags like this to Cargo: rustc-link-lib=static=libname in case of static libs and rustc-link-lib=libname in case of non-static libs. If there is some ambiguous cases, please enlighten me.

On Windows etc. passing such a flag (let's say, --all-static) should just make the build fail. As it should when trying to link with glibc – that doesn't support static linking either.

[golddranks]

To be specific, the case I'm talking about here is when building with --target=x86_64-unknown-linux-musl. (I don't know about other static targets.) At the moment, one of the defining motivations to build with musl is to get a fully static build. However, because the compiler accepts silently non-static linking flags, configuring such a build might be needlessly hard. Edit: At the moment, it involves building with --verbose and inspecting the flags passed to rustc – and that's if you know what you're doing. And if you're building in a virtualized environment (let's say that your dev environment makes it hard to install and configure libs for cross-compiling), that's even more frustrating since you have to do debugging in a foreign environment.

[est31]

Its would be a nice feature, but I think for consistency sake it should be part of a greater effort about unifying how to pass various parameters to build.rs and how their expected behaviour should be.

[binarycrusader]

Like @golddranks implies, this flag seems of limited utility. As soon as only one of a given dependency is only available in dynamic form, the flag is useless. As another example, like Windows, the system libraries on Solaris are only available dynamically (such as libc).

In short, I think such a flag could be safely applied to all rust libraries, but not any external dependencies (C libraries, system libraries, etc.). Keep in mind too that rust isn't the one doing the linking; the linker can (behind rust's back) decide to dynamically link something additional anyway (as gcc does in some cases). Ultimately, the only way to determine whether there is anything dynamically linked is to check the binary afterwards for additional dynamic dependencies.

If this functionality really is required, then I'd suggest that a simple flag such as the one proposed here is not quite sufficient; it really needs more nuance; such as --all-static=rust, --all-static=system, and so on. With that in mind, I'd tend to agree with @est31; a more holistic approach is needed.

[golddranks]

I beg to differ with your first point: the flag is most useful precisely when only the dynamic library is available, because that's a condition that one might want to guard against, but currently the compiler lets it pass silently. It was my original motivation for suggesting this.

[binarycrusader]

The point I was attempting to make is that if the only way to build a given item on a given platform is by dynamic linking, then the flag will never work, and so is of limited utility. (At least, not without some compromise, as I suggest later.)

I agree wholeheartedly with some way of ensuring that a given set of dependencies are static, but I don't think a blanket flag is going to work very well -- it doesn't account for the realities of different targets.

Instead of a flag, I suspect something in Cargo.toml would be more appropriate. Basically, what I think you really want is a declaration of intent on a per-target, per-dependency basis in your Cargo.toml.

For example, in your case, you know what when you are targeting musl, you want all system libraries to be static, and then for each of your rust dependencies, you want them to be static as well.

However, if you're not targeting musl, you might be willing to accept dynamic system libs, but still want to assert that all of your rust dependencies are static. That then allows you to match the reality of some platforms being static and some not when it comes to system libs.

The alternative would be to treat system libraries as special, and just have rust be smart enough to realise that if you're targeting Windows or Solaris, system libraries will always be dynamic, so should not cause failure of --all-static. However, when you're targeting musl, it could fail if it could not statically link the system libraries.

[codyps]

@binarycrusader listing all the required link type for all transitive dependencies of a crate is unlikely to be reasonable. Splitting the world between rust and system is too coarse a separation.

Agreed that a flag like this doesn't solve all situations, and resolving that likely means having better handling for informing build.rs scripts about requirements of other crates.

Would still be a useful thing to be able to specify "final output must be static".

[binarycrusader]

@jmesmon I had already considered the transitive closure of dependencies and had assumed that it would apply based on the parent dependency. That is, if I declare that I require the foo crate to be static, then the foo crate and its transitive closure of dependencies must also be static. With the only exception being that any system dependencies encountered would fall under the top-level rules.

[codyps]

I don't think that's a reasonable assumption. I'm struggling to think of a case where it would make sense for a crate to only require some of it's dependency sub-graphs to be static. It also doesn't account for what happens when those sub-graphs are connected. Is there a use case you're thinking of that is covered by your proposal?