Calling HashMap::new() from Dynamically loaded lib, causes Segfault, Illegal Instruction

[ashleysommer]

I was running across this issue in my project last week.
See Stackoverflow question here and reddit.com/r/rust thread here

Basically, calling HashMap::new() inside my runtime-dynamically-loaded plugin lib is causing an unwind, and then rust segfaults while doing the unwind. Running the application on the command line gives Illegal Instruction output, running inside gdb gives:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff51a0c60 in ?? ()

I've produced a minimal test case:
main.rs

use std::dynamic_lib::DynamicLibrary;
use std::mem;

fn main()
{
    let p1 = match DynamicLibrary::open(Some("libplugin.so")) {
            Ok(lib) => lib,
            Err(error) => panic!("Could not load the library: {}", error)
    };
    let s1: extern "Rust" fn() = unsafe {
        match p1.symbol("init") {
            Err(error) => panic!("Could not load function init: {}", error),
            Ok(init) => mem::transmute::<*mut u8, _>(init)
        }
    };
    s1();
}

plugin.rs

use std::collections::hashmap::{HashMap};
use std::collections::treemap::{TreeMap};

#[no_mangle]
pub fn init()
{
     let h:HashMap<&str, &str> = HashMap::new();
     //let t:TreeMap<&str, &str> = TreeMap::new();
     println!("Loaded!");
}

Changing HashMap to TreeMap in plugin.rs causes the segfault to go away, and the program runs correctly.

Im using the latest Rust master codebase, compiled on linux x64.

[sfackler]

Looks like the runtime isn't getting initialized properly. The segfault is occurring after the plugin panics due to an error in the task local RNG:

#0  0x00007ffff5ee8690 in rust_panic () from /usr/local/lib/librustrt-4e7c5e5c.so
#1  0x00007ffff5ee8e6a in unwind::begin_unwind_inner::h043457e0ce41796ddSd () from /usr/local/lib/librustrt-4e7c5e5c.so
#2  0x00007ffff5ee8b29 in unwind::begin_unwind_fmt::h33cb99eebdc73181FPd () from /usr/local/lib/librustrt-4e7c5e5c.so
#3  0x00007ffff621f6ad in rand::task_rng::h87e31a49ab53b7b4Gib () from /usr/local/lib/libstd-4e7c5e5c.so
#4  0x00007ffff65fc34b in hash::RandomSipHasher::new::h7ac2f0f050fe1602c5f () from ./libplugin.so
#5  0x00007ffff65fc2e3 in collections::hashmap::map::HashMap$LT$K$C$$x20V$C$$x20RandomSipHasher$GT$::new::h6355596007518334254 () from ./libplugin.so
#6  0x00007ffff65fc1e9 in init () from ./libplugin.so
#7  0x000055555555e44f in main::h77f94290d38a5ccegaa ()
#8  0x0000555555594d3d in start::closure.8528 ()
#9  0x00005555555acbcc in rust_try_inner ()
#10 0x00005555555acbb6 in rust_try ()
#11 0x00005555555aa233 in unwind::try::hac84dac916339a9cxGd ()
#12 0x00005555555aa10c in task::Task::run::h3855d187b4c0788fnMc ()
#13 0x0000555555594b7f in start::h6e41679d4b8806beUhe ()
#14 0x00005555555949b6 in lang_start::h1776e07af04107b5dhe ()
#15 0x000055555555e4df in main ()

[ashleysommer]

Thanks for looking at this. That call stack clearly shows it is choking in the rng function, which is more than I was able to deduce. Can I ask how you generated that call stack/backtrace? The best I was able to do was step through the code in my IDE (with gdb backend), even though I was using rust libs compiled with debug symbols, it would just step straight from HashMap::new() to the exception in rust_try(), with no steps in between.

To the issue at hand, I may be in way over my head, but how is creating a HashMap inside the plugin different than creating it in the main application? Why is the runtime in a different state (not initialized properly) in the plugin, should it not be in the same state as the calling code?

[sfackler]

I inserted a breakpoint in GDB at the rust_panic function, which is called just before unwinding begins.

I have no idea why it's breaking. Compiler plugins are loaded in the same way, and they can use hashmaps without any problems. It might be that there's some thing that blows up if it doesn't happen in the main binary the first time?

[ashleysommer]

Ok, I've managed to debug it a bit further with some manual GDB commands.
It looks like OsRng::new() (called by task_rng()) is not able to open /dev/urandom. It fails with the message 'Invalid Argument'.

Here is the crude gdb output:

std::rand::os::imp::OsRng::new () at src/libstd/macros.rs:321
321     ($e:expr) => (match $e { Ok(e) => e, Err(e) => return Err(e) })
(gdb) print $e
$3 = void
(gdb) print e
$4 = {kind = {{RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {
      RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {
      RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {
      RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {
      RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {
      RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput}, {RUST$ENUM$DISR = InvalidInput, 
      22}, {RUST$ENUM$DISR = InvalidInput}}, desc = {data_ptr = 0x7ffff5662af0 <str26560> "couldn't open file; path=; mode=; access=", 
    length = 18}, detail = {{RUST$ENUM$DISR = Some}, {RUST$ENUM$DISR = Some, {vec = {
          ptr = 0x7ffff4425000 "invalid argument; path=/dev/urandom; mode=open; access=read", '\245' <repeats 69 times>, len = 59, 
          cap = 128}}}}}

[sfackler]

Interesting! Here's a smaller version of plugin.rs that causes the same issue:

use std::io::fs::File;

#[no_mangle]
pub fn init()
{
    let _ = File::open(&Path::new("/dev/urandom")).unwrap();
}

[huonw]

Does it fail with other files?

[sfackler]

@huonw yep

[ashleysommer]

Ok, maybe need to rename the title of this issue, its got nothing to do with HashMap now.

[bkoropoff]

Are you building the main program with -C prefer-dynamic? Otherwise you're going to have libplugin.so linking libstd dynamically and main linking it statically, and weird things are going to happen after dlopen

[sfackler]

Oh, right! -C prefer-dynamic makes everything work as expected for me.

[ashleysommer]

Ohh. Thanks! I was incorrectly assuming that rustc uses dynamic libs automatically. No wonder my compiled file sizes were bigger than I expected. Thanks again guys!