copy any file from stage0/lib to stage0-sysroot/lib
With the LLVM 18 upgrade, the name of the LLVM library has been changed to something like `libLLVM.so.18.1-rust-1.78.0-beta`, which `is_dylib` function cannot determine as it only looks whether files are ending with ".so" or not.
This change resolves this problem by no longer doing that ".so" check, as we need all files from the stage0/lib as they are all dependency of rustc anyway.
Fixes#122913
With the LLVM 18 upgrade, the name of the LLVM library has been changed to something like
`libLLVM.so.18.1-rust-1.78.0-beta`, which `is_dylib` function cannot determine as it only
looks whether files are ending with ".so" or not.
This change resolves this problem by no longer doing that ".so" check, as we need all files
from the stage0/lib as they are all dependency of rustc anyway.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
fix(bootstrap/dist): use versioned dirs when vendoring
Currently, if you attempt to run ui tests in a vendored build, you will
see this failure
```
---- [ui] tests/ui/issues/issue-21763.rs stdout ----
diff of stderr:
8 = note: required because it appears within the type `(Rc<()>, Rc<()>)`
9 = note: required for `hashbrown::raw::RawTable<(Rc<()>, Rc<()>)>` to implement `Send`
10 note: required because it appears within the type `hashbrown::map::HashMap<Rc<()>, Rc<()>, RandomState>`
- --> $HASHBROWN_SRC_LOCATION
+ --> /rust/deps/hashbrown/src/map.rs:190:12
12 note: required because it appears within the type `HashMap<Rc<()>, Rc<()>>`
13 --> $SRC_DIR/std/src/collections/hash/map.rs:LL:COL
14 note: required by a bound in `foo`
```
This happens because the code that attempts to remap
`HASHBROWN_SRC_LOCATION` expects it to be under `hashbrown-$version`,
which is the case in a normal cargo registry, but not when vendoring, where
by default crates may not have the version in their directory name.
This change passes `--versioned-dirs` to `cargo vendor` to enforce that
every crate includes the version in the subdir name, which fixes the ui
test and brings `--enable-vendor` builds closer to normal ones.
Fix nix patching for LLVM 18
LLVM 18 now ships `libLLVM*.so.*`, so `.so` is not the sole extension anymore, which breaks the dylib detection. Oops! Adjust it to only search for `.so` somewhere.
fixes#122906
The dead_code lint was previously eroneously missing this dead code.
Since this lint bug has been fixed, the unused field need to be removed
or marked as `#[allow(dead_code)]`.
Given that this struct is deserialized without #[serde(deny_unknown_fields)]
it is ok to simply delete the never read fields.
LLVM 18 now ships `libLLVM*.so.*`, so `.so` is not the sole extension
anymore, which breaks the dylib detection. Oops! Adjust it to only
search for `.so` somewhere.
Currently, if you attempt to run ui tests in a vendored build, you will
see this failure
```
---- [ui] tests/ui/issues/issue-21763.rs stdout ----
diff of stderr:
8 = note: required because it appears within the type `(Rc<()>, Rc<()>)`
9 = note: required for `hashbrown::raw::RawTable<(Rc<()>, Rc<()>)>` to implement `Send`
10 note: required because it appears within the type `hashbrown::map::HashMap<Rc<()>, Rc<()>, RandomState>`
- --> $HASHBROWN_SRC_LOCATION
+ --> /rust/deps/hashbrown/src/map.rs:190:12
12 note: required because it appears within the type `HashMap<Rc<()>, Rc<()>>`
13 --> $SRC_DIR/std/src/collections/hash/map.rs:LL:COL
14 note: required by a bound in `foo`
```
This happend because the code that attempts to remap
`HASHBROWN_SRC_LOCATION` expects it to be under `hashbrown-$version`,
which is the case in a normal cargo registry, but not when vendor, where
by default crates may not have the version in their directory name.
This change passes `--versioned-dirs` to `cargo vendor` to enforce that
every crate includes the version in the subdir name, which fixes the ui
test and brings `--enable-vendor` builds closer to normal ones.
This is implemented with the freshly-released Wasmtime 19 and should
prevent beta breakage from wasm tests that was observed and fixed
in #122640 again.
various clippy fixes
We need to keep the order of the given clippy lint rules before passing them.
Since clap doesn't offer any useful interface for this purpose out of the box,
we have to handle it manually.
Additionally, this PR makes `-D` rules work as expected. Previously, lint rules were limited to `-W`. By enabling `-D`, clippy began to complain numerous lines in the tree, all of which have been resolved in this PR as well.
Fixes#121481
cc `@matthiaskrgr`
Update the minimum external LLVM to 17
With this change, we'll have stable support for LLVM 17 and 18.
For reference, the previous increase to LLVM 16 was #117947.
Enable frame pointers for the standard library
There's been a few past experiments for enabling frame pointers for all our artifacts. I don't think that frame pointers in the distributed compiler are nearly as useful as frame pointers in the standard library. Our users are much more likely to be profiling apps written in Rust than they are profiling the Rust compiler.
So yeah it would be cool to have frame pointers in the compiler, but much more of the value is having them on the precompiled standard library. That's what this PR does.
Previously, when passing lint rules manually using `x clippy ..`, ignored lints would
override manual ones. This change corrects the order by passing ignored lints after the
manual ones.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
We need to keep the order of the given clippy lint rules before passing them.
Since clap doesn't offer any useful interface for this purpose out of the box,
we have to handle it manually.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
run change tracker even when config parse fails
Please note that we are currently validating the build configuration on two entry points (e.g., profile validation is handled on the python side), and change tracker system is handled on the rust side. Once #94829 is completed (scheduled for 2024), we will be able to handle this more effectively.
Fixes#121756
mir-opt: always run tests for the current target
Currently, `./x.py test tests/mir-opt` runs only the tests for the current target, and `./x.py test tests/mir-opt --bless` runs tests for a representative set of targets. That representative set does not include the current target however, which means `--bless` can succeed when tests fail without it. This PR ensures we run the current target always.
Fixes https://github.com/rust-lang/rust/issues/122292
cc ``@RalfJung``
bootstrap: Don't name things copy that are not copies
The bootstrap copy methods don't actually copy, they just hard link. Simply lying about it being "copying" can be very confusing! (ask me how I know!).
I'm not sure whether the name I chose is the ideal name, but it's definitely better than before.
prevent notifying the same changes more than once
Prevents re-reporting of previously notified changes by using the .last-warned-change-id value for change detection.
Resolves#122344
The bootstrap copy methods don't actually copy, they just hard link.
Simply lying about it being "copying" can be very confusing! (ask me how
I know!).
Previously, doing `x test compiler/*` would fail the build due to missing std.
This change ensures that it is prepared.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Properly rebuild rustbooks
Fixes#122367
If the book was out of date but the tool was up to date, this would evaluate to `!(false || true)` == `!true` == `false` and not rebuild.
bootstrap: Don't eagerly format verbose messages
We `format!` a lot of messages which are only used when we are at some level of verbosity - do this lazily instead
[bootstrap] Move the `split-debuginfo` setting to the per-target section
As described in #112406, bootstrap currently applies the global `split-debuginfo` setting to all artifacts, irrespective of their target triple.
This doesn't cause problems when the "build" triple defaults `split-debuginfo` to `off` (as is the case on Linux, for example).
However, when the "build" triple has `split-debuginfo` enabled and additional target triples are configured, then artifacts for the additional triples will also be built with `split-debuginfo` (despite not necessarily supporting `split-debuginfo`).
#112406 mentions `riscv32imc-unknown-none-elf` as one target where this happens, and I've run into this with Wasm as well.
This PR does **not** implement `@ehuss's` suggestion that "bootstrap not try to guess how to configure split-debuginfo, and instead use cargo profiles to set it", because that seemed like a lot more significant change.
---
After this PR, anyone explicitly setting `rust.split-debuginfo` should update their configuration to specify the setting in the `target.<triple>` section, though `rust.split-debuginfo` will still be honored for the "build" triple for now.
This PR changes the behavior when `rust.split-debuginfo` was **not** explicitly set **and** bootstrap is configured to cross-compile to a triple that has a different `split-debuginfo` than the "build" triple.
---
If there's a reasonable way to add additional tests for this, please let me know (I didn't find any tests checking cargo arguments in [`builder/tests.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/src/core/builder/tests.rs)).
Test wasm32-wasip1 in CI, not wasm32-unknown-unknown
This commit changes CI to no longer test the `wasm32-unknown-unknown` target and instead test the `wasm32-wasip1` target. There was some discussion of this in a [Zulip thread], and the motivations for this PR are:
* Runtime failures on `wasm32-unknown-unknown` print nothing, meaning all you get is "something failed". In contrast `wasm32-wasip1` can print to stdout/stderr.
* The unknown-unknown target is missing lots of pieces of libstd, and while `wasm32-wasip1` is also missing some pieces (e.g. threads) it's missing fewer pieces. This means that many more tests can be run.
Overall my hope is to improve the debuggability of wasm failures on CI and ideally be a bit less of a maintenance burden.
This commit specifically removes the testing of `wasm32-unknown-unknown` and replaces it with testing of `wasm32-wasip1`. Along the way there were a number of other archiectural changes made as well, including:
* A new `target.*.runtool` option can now be specified in `config.toml` which is passed as `--runtool` to `compiletest`. This is used to reimplement execution of WebAssembly in a less-wasm-specific fashion.
* The default value for `runtool` is an ambiently located WebAssembly runtime found on the system, if any. I've implemented logic for Wasmtime.
* Existing testing support for `wasm32-unknown-unknown` and Emscripten has been removed. I'm not aware of Emscripten testing being run any time recently and otherwise `wasm32-wasip1` is in theory the focus now.
* I've added a new `//@ needs-threads` directive for `compiletest` and classified a bunch of wasm-ignored tests as needing threads. In theory these tests can run on `wasm32-wasi-preview1-threads`, for example.
* I've tried to audit all existing tests that are either `ignore-emscripten` or `ignore-wasm*`. Many now run on `wasm32-wasip1` due to being able to emit error messages, for example. Many are updated with comments as to why they can't run as well.
* The `compiletest` output matching for `wasm32-wasip1` automatically uses "match a subset" mode implemented in `compiletest`. This is because WebAssembly runtimes often add extra information on failure, such as the `unreachable` instruction in `panic!`, which isn't able to be matched against the golden output from native platforms.
* I've ported most existing `run-make` tests that use custom Node.js wrapper scripts to the new run-make-based-in-Rust infrastructure. To do this I added `wasmparser` as a dependency of `run-make-support` for the various wasm tests to use that parse wasm files. The one test that executed WebAssembly now uses `wasmtime`-the-CLI to execute the test instead. I have not ported over an exception-handling test as Wasmtime doesn't implement this yet.
* I've updated the `test` crate to print out timing information for WASI targets as it can do that (gets a previously ignored test now passing).
* The `test-various` image now builds a WASI sysroot for the WASI target and additionally downloads a fixed release of Wasmtime, currently the latest one at 18.0.2, and uses that for testing.
[Zulip thread]: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Have.20wasm.20tests.20ever.20caused.20problems.20on.20CI.3F/near/424317944
