The submodule was causing issues in rust-lang/rust, so eliminiate it here. `build-musl` is also removed from `libm-test`'s default features so the crate doesn't need to be built by default.
