Fix collision doc tests randomly failing.
This fixes some tests that were randomly failing on CI. The cause is that #9419 added a remove_dir_all on the `doc` directory. However, if two jobs are trying to write to that directory at the same time, this can cause errors. The failure rate is low (a little over 1%), and I was unable to reproduce locally (only on GitHub's CI and only on the Windows job).
The solution is to run the jobs with -j1 so they run serially.
I only saw errors for `collision_doc_sources`, but to be on the safe side I added j1 to similar tests.
Fix dep-info files emitting paths relative to deps' roots
Sample `shoo.d` file prior to this change is below, note the `build.rs`
at the end, which was not from my package.
From booping the debugger, I found this was coming from
`compiler_builtins`. This is not really their bug though: if a build.rs
asks for rerun-if-changed on some crate relative path, this will happen
in general. So I've fixed it in Cargo and added a test to prevent it
regressing.
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s build.rs
```
This change fixes it so it's like:
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.39/build.rs /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/log-0.4.14/build.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s
```
Fix rebuild issues with rustdoc.
This fixes two issues related to rebuilds with rustdoc:
* Switching features when running `cargo doc` would result in Cargo not rebuilding the documentation. This is because it was keeping the fingerprints in separate directories based on the features used. However, the rustdoc output isn't keyed off the metadata hash, so although the old fingerprint seemed "up to date", in reality the documentation was rewritten and needs to be rebuilt. The solution is to use a simplified hash for the fingerprint directory name.
* Removing items does not remove the files from the doc directory. This changes it to clear the package's doc directory before running rustdoc, to ensure any stale files are removed.
I'm a little concerned about potential performance impact of running `remove_dir_all`, but I think it shouldn't be too bad?
Fixes#7370
Sample `shoo.d` file prior to this change is below, note the `build.rs`
at the end, which was not from my package.
From booping the debugger, I found this was coming from
`compiler_builtins`. This is not really their bug though: if a build.rs
asks for rerun-if-changed on some crate relative path, this will happen
in general. So I've fixed it in Cargo and added a test to prevent it
regressing.
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s build.rs
```
This change fixes it so it's like:
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.39/build.rs /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/log-0.4.14/build.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s
```
Expose build.target .cargo/config setting as packages.target in Cargo.toml
Hey!
I'm trying to do my first cargo contribution by implementing per-crate target settings as per [the irlo thread](https://internals.rust-lang.org/t/proposal-move-some-cargo-config-settings-to-cargo-toml/13336) ; and I think I have a draft that looks good-ish (the root units returned by `generate_targets` have the right kinds set).
Closes#7004
**_Edit: the below problem description is now solved in the latest version of this PR, please ignore_**
But for some reason running on a test project now blocks on `Blocking waiting for file lock on build directory` and I have literally no idea how my changes could trigger this… would anyone have an idea of how the changes could lead to infinitely blocking there? (I already tried cargo clean just in case and it didn't appear to help)
FWIW, the output that looks hopeful to me is, on my testbed workspace:
```
Root units [out of generate_targets] are [...]:
- package ‘smtp-client’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-server’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-rpc’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config-example’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config-example’ with kind ‘Target(CompileTarget { name: "wasm32-unknown-unknown" })’
- package ‘smtp-queue’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-server-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config’ with kind ‘Target(CompileTarget { name: "wasm32-unknown-unknown" })’
- package ‘yuubind’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-queue-fs’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
```
(where both `yuubind-config` and `yuubind-config-example` are being configured to be `wasm32-unknown-unknown` and the other ones stay as host).
Interestingly enough, if I remove the `target` setting from `yuubind-config` (and leave it on `yuubind-config-example`) then it does no longer block on waiting for file lock on build directory, even though it does not actually compile with `wasm32-unknown-unknown`. And it does appear to correctly build yuubind-config-example as wasm32.
My investigation shows that it appears to happen iff there is a package with `package.target` being set that has dependencies.
This most likely is a bug in my code (eg. I build only the root units and not the whole unit graph maybe?), and am going to keep investigating it as such, but maybe someone would already know how dependency resolution could interact with build lock acquisition and give me hints?
Anyway, thank you all for all you do cargo!
Fix loading `branch=master` patches in the v3 lock transition
This commit fixes an issue pointed out during #9352 where in the v2->v3
lock file transition (currently happening on nightly) Cargo will not
correctly use the previous lock file entry for `[patch]` directives that
point to git dependencies using `branch = 'master'` explicitly. The
reason for this is that Cargo previously, with the v2 format, considered
`branch=master` and `DefaultBranch` to be equivalent dependencies. Now
that Cargo treats those as distinct resolve nodes we need to load lock
files that use `DefaultBranch` and transparently use those for
`branch=master` dependencies.
These lock file nodes do not naturally unify so we have to go out of our
way to get the two to line up in modern Cargo. This was previously done
for the lock file at large, but the previous logic didn't take `[patch]`
into account. Unfortunately almost everything to do with `[patch]` and
lock files is pretty complicated, and this is no exception. The fix here
is wordy, verbose, and quite subtle in how it works. I'm pretty sure it
does work though and I think that this should be good enough to at least
transition most users off the v2 lock file format. Once this has baked
in Cargo for some time (on the scale of a year) I would hope that we
could just remove this logic since it's only really here for a
transitionary period.
Closes#9352
This commit fixes an issue pointed out during #9352 where in the v2->v3
lock file transition (currently happening on nightly) Cargo will not
correctly use the previous lock file entry for `[patch]` directives that
point to git dependencies using `branch = 'master'` explicitly. The
reason for this is that Cargo previously, with the v2 format, considered
`branch=master` and `DefaultBranch` to be equivalent dependencies. Now
that Cargo treats those as distinct resolve nodes we need to load lock
files that use `DefaultBranch` and transparently use those for
`branch=master` dependencies.
These lock file nodes do not naturally unify so we have to go out of our
way to get the two to line up in modern Cargo. This was previously done
for the lock file at large, but the previous logic didn't take `[patch]`
into account. Unfortunately almost everything to do with `[patch]` and
lock files is pretty complicated, and this is no exception. The fix here
is wordy, verbose, and quite subtle in how it works. I'm pretty sure it
does work though and I think that this should be good enough to at least
transition most users off the v2 lock file format. Once this has baked
in Cargo for some time (on the scale of a year) I would hope that we
could just remove this logic since it's only really here for a
transitionary period.
Closes#9352
Fix disagreement about lockfile ordering on stable/nightly
This commit fixes an issue where the order of packages serialized into a
lock file differs on stable vs nightly. This is due to a bug introduced
in #9133 where a manual `Ord` implementation was replaced with a
`#[derive]`'d one. This was an unintended consequence of #9133 and means
that the same lock file produced by two different versions of Cargo only
differs in what order items are serialized.
With #9133 being reverted soon on the current beta channel this is
intended to be the nightly fix for #9334. This will hopefully mean that
those projects which don't build with beta/nightly will remain
unaffected, and those affected on beta/nightly will need to switch to
the new nightly ordering when it's published (which matches the current
stable). The reverted beta will match this ordering as well.
Closes#9334
Don't give a hard error when the end-user specifies RUSTC_BOOTSTRAP=crate_name
Fixes https://github.com/rust-lang/cargo/issues/9362.
The whole point of https://github.com/rust-lang/rust/pull/77802/ was to allow specifying this granularly, giving a hard error defeats the point.
I didn't know how to check what targets were reverse-dependencies of build.rs, so I just unconditionally use the library name (and give a hard error for anything else, even if it's the name of one of the binaries). End-users can still opt-in with RUSTC_BOOTSTRAP=1, and no public binaries use RUSTC_BOOTSTRAP=1, so I don't think this a big deal in practice.
<details><summary>Script to verify all crates using RUSTC_BOOTSTRAP=1 have a library</summary>
```sh
curl https://pastebin.com/raw/fGQ97xP6 | cut -d / -f1 | grep -v shnatsel | grep -v cargo- | sed 's#-\([0-9]\)#/\1#' | xargs -i curl -s -I -L "https://docs.rs/{}/" -w "%{http_code}\n" -o/dev/null
```
It should output 20 200s in a row.
</details>
r? `@ehuss` cc `@mark-simulacrum`
I don't know what cargo's policy is for backports, but this should be backported to 1.52.
Fix#9350 (cargo build -Z help is missing options)
> Do not merge yet, some options are still undocumented.
Fix#9350 (cargo build -Z help is missing options)
Add a procedural macro to declare `CliUnstable` struct and provide help messages instead of hard-coding help in `src/bin/cargo/cli.rs`
> Flags documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html
Feedback welcome
> Do not merge yet, some options are still undocumented.
Fix#9350 (cargo build -Z help is missing options)
Add a procedural macro to declare `CliUnstable` struct and provide help messages instead of hard-coding help in `src/bin/cargo/cli.rs`
> Flags documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html
Feedback welcome
This commit fixes an issue where the order of packages serialized into a
lock file differs on stable vs nightly. This is due to a bug introduced
in #9133 where a manual `Ord` implementation was replaced with a
`#[derive]`'d one. This was an unintended consequence of #9133 and means
that the same lock file produced by two different versions of Cargo only
differs in what order items are serialized.
With #9133 being reverted soon on the current beta channel this is
intended to be the nightly fix for #9334. This will hopefully mean that
those projects which don't build with beta/nightly will remain
unaffected, and those affected on beta/nightly will need to switch to
the new nightly ordering when it's published (which matches the current
stable). The reverted beta will match this ordering as well.
Closes#9334
Don't re-use rustc cache when RUSTC_WRAPPER changes
We cache initial `rustc --version` invocations, to speed up noop builds.
We check the mtime of `rustc` to bust the cache if the complier changed.
However, before this PR, we didn't look at mtimes of `RUSTC_WRAPPER` /
`RUSTC_WORKSPACE_WRAPPER`, so we could've re-use old cache with new
wrapper.
This doesn't work because it uses the name of the build script, which is
always build_script_build. I'm not sure what to change it to - the name
of the library crate could be different than the name of the package,
and there could be multiple different crates being compiled in the same
package.
We cache initial `rustc --version` invocations, to speed up noop builds.
We check the mtime of `rustc` to bust the cache if the complier changed.
However, before this PR, we didn't look at mtimes of `RUSTC_WRAPPER` /
`RUSTC_WORKSPACE_WRAPPER`, so we could've re-use old cache with new
wrapper.
"exit code" is wrong terminology on Unix. I am trying to fix this in
Rust stdlib in
https://github.com/rust-lang/rust/pull/83462
but this currently breaks the cargo test suite.
See that MR for full explanation of the change.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Default macOS targets to `unpacked` debuginfo
This commit continues the work from #9112 to enable `unpacked` split
debuginfo on macOS targets by default. This has been discussed on [internals]
for awhile now and no breakage has emerged while significant speedups
have. This is expected to be a compile-time and `target`-directory size
win for almost all macOS Rust projects.
While breakage is possible it's possible to mitigate this with
project-local or global cargo configuration of the `dev` and `test` profiles.
[internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
This commit continues the work from #9112 to enable `unpacked` split
debuginfo on macOS targets by default. This has been discussed on [internals]
for awhile now and no breakage has emerged while significant speedups
have. This is expected to be a compile-time and `target`-directory size
win for almost all macOS Rust projects.
While breakage is possible it's possible to mitigate this with
project-local or global cargo configuration of the `dev` and `test` profiles.
[internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
This commit fixes an issue where packages which specify `resolver = '2'`
cannot be packaged if they also have a `package.metadata` table. The
issue is that the `toml` serialization implementation serializes fields
in order which requires that tables be emitted last.
