#[global_allocator] in dependent crate can be optimized away

[jorendorff]

jemallocator-global is a library intended to make jemalloc your allocator. It works by declaring a static allocator with #[global_allocator].

However, a downstream crate linked with jemallocator-global nonetheless uses the system malloc on MacOS X.

After experimenting a bit with #[global_allocator], it's behaving as though rustc optimizes away a whole crate if you don't seem to be using it; and it doesn't detect that the global allocator is always used.

[jorendorff]

It looks like extern crate jemallocator_global; might be enough to get the allocator to be linked and used. So maybe this is intentional behavior? It's surprising from a user's perspective, though.

[gnzlbg]

I think that #[globall_allocator] should maybe imply #[used], cc @alexcrichton

[alexcrichton]

@jorendorff to confirm you added jemallocator-global = "..." to Cargo.toml, but the application didn't work as expected to use jemalloc? And then adding extern crate jemallocator_global; fixed the issue?

[jorendorff]

First question, yes.

Second question, no. Instead, I made a separate project to play with, and in that project extern crate has this effect. I'll post that as a repo in a sec.

[mati865]

a downstream crate linked with jemallocator-global nonetheless uses the system malloc on MacOS X.

AFAIK jemallocator-global and #[globall_allocator] are meant to be used in the topmost binary (the one which depends on other crates). Or am I misunderstanding?

[jorendorff]

Here's the repo that shows the bug: https://github.com/jorendorff/rust-lang-bug-64402/

The allocator in this repo panics on alloc, so it's easy to tell if it was used or not.

Adding #[used] to the allocator didn't fix the issue.

[jorendorff]

AFAIK jemallocator-global and #[globall_allocator] are meant to be used in the topmost binary (the one which depends on other crates). Or am I misunderstanding?

jemallocator-global contains a #[global_allocator]: https://docs.rs/jemallocator-global/0.3.2/src/jemallocator_global/lib.rs.html#35

[alexcrichton]

Ok, thanks for the clarification! In that case, yes, #[used] isn't going to fix this. At this time there's no way to fix this other than using extern crate I believe.

Most crates when linked don't actually have any side effect (e.g. just a bland rlib), but some crates (like the global allocator) when linked actually affect how the compiler structures things. To be conservative the --extern flag on the command line doesn't actually link the crate, it just provides it as a candidate for linkage. If you use anything from the crate that's considered to link it, but if you don't actually use it then the crate isn't linked at all. In this case adding extern crate is explicitly linking the crate. You could perhaps also fix this as use jemallocator_global somewhere (or some path from the crate).

As a bug the compiler could perhaps give special treatment to --extern arguments that have a #[global_allocator], but that's also a tricky path to go down. This is sort of just an unfortunate bug in general, and I'm not sure it has a great solution.

[gnzlbg]

Is this also a bug that, e.g., #[no_std] crates defining a #[panic_handler] that are put in the dependency graph but aren't "used" would have ?

[mati865]

@jorendorff this is what I mean by topmost crate: https://gist.github.com/mati865/05f96512240e974dec0ca9740316bde6 the same thing should apply for jemallocator-global crate.

[alexcrichton]

@gnzlbg I don't really know for sure, I'd try it out. If that works then #[global_allocator] is missing treatment that #[panic_handler] has. If it doesn't work then it's probably the same issue.