Build host dependencies with opt-level 0 by default
This commit updates Cargo's build of host dependencies to build them
with optimization level 0 by default instead of matching the profile of
the final binary.
Since Cargo's inception build dependencies have, by default, been built
in a profile that largely matches the profile of the final target
artifact. Build dependencies, however, rarely actually need to be
optimized and are often executing very small tasks, which means that
optimizing them often wastes a lot of build time. A great example of
this is procedural macros where `syn` and friends are pretty heavyweight
to optimize, and the amount of Rust code they're parsing is typically
quite small, so the time spent optimizing rarely comes as a benefit.
The goal of this PR is to improve build times on average in the
community by not spending time optimizing build dependencies (build
scripts, procedural macros, and their transitive dependencies). The PR
will not be a universal win for everyone, however. There's some
situations where your build time may actually increase:
* In some cases build scripts and procedural macros can take quite a
long time to run!
* Cargo may not build dependencies more than once if they're shared with
the main build. This only applies to builds without `--target` where
the same crate is used in the final binary as in a build script.
In these cases, however, the `build-override` profile has existed for
some time know and allows giving a knob to tweak this behavior. For
example to get back the previous build behavior of Cargo you would
specify, in `Cargo.toml`:
[profile.release.build-override]
opt-level = 3
or you can configure this via the environment:
export CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_OPT_LEVEL=3
There are two notable features we would like to add in the future which
would make the impact of a change like this smaller, but they're not
implemented at this time (nor do we have concrete plans to implement
them). First we would like crates to have a way of specifying they
should be optimized by default, despite default profile options. Often
crates, like lalrpop historically, have abysmal performance in debug
mode and almost always (even in debug builds) want to be built in
release mode. The second feature is that ideally crate authors would be
able to tell Cargo to minimize the number of crates built, unifying
profiles where possible to avoid double-compiling crates.
At this time though the Cargo team feels that the benefit of changing
the defaults is well worth this change. Neither today nor directly after
this change will be a perfect world, but it's hoped that this change
makes things less bad!
This commit updates Cargo's build of host dependencies to build them
with optimization level 0 by default instead of matching the profile of
the final binary.
Since Cargo's inception build dependencies have, by default, been built
in a profile that largely matches the profile of the final target
artifact. Build dependencies, however, rarely actually need to be
optimized and are often executing very small tasks, which means that
optimizing them often wastes a lot of build time. A great example of
this is procedural macros where `syn` and friends are pretty heavyweight
to optimize, and the amount of Rust code they're parsing is typically
quite small, so the time spent optimizing rarely comes as a benefit.
The goal of this PR is to improve build times on average in the
community by not spending time optimizing build dependencies (build
scripts, procedural macros, and their transitive dependencies). The PR
will not be a universal win for everyone, however. There's some
situations where your build time may actually increase:
* In some cases build scripts and procedural macros can take quite a
long time to run!
* Cargo may not build dependencies more than once if they're shared with
the main build. This only applies to builds without `--target` where
the same crate is used in the final binary as in a build script.
In these cases, however, the `build-override` profile has existed for
some time know and allows giving a knob to tweak this behavior. For
example to get back the previous build behavior of Cargo you would
specify, in `Cargo.toml`:
[profile.release.build-override]
opt-level = 3
or you can configure this via the environment:
export CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_OPT_LEVEL=3
There are two notable features we would like to add in the future which
would make the impact of a change like this smaller, but they're not
implemented at this time (nor do we have concrete plans to implement
them). First we would like crates to have a way of specifying they
should be optimized by default, despite default profile options. Often
crates, like lalrpop historically, have abysmal performance in debug
mode and almost always (even in debug builds) want to be built in
release mode. The second feature is that ideally crate authors would be
able to tell Cargo to minimize the number of crates built, unifying
profiles where possible to avoid double-compiling crates.
At this time though the Cargo team feels that the benefit of changing
the defaults is well worth this change. Neither today nor directly after
this change will be a perfect world, but it's hoped that this change
makes things less bad!
Fix freshness checks for build scripts on renamed dirs
This commit fixes an issue in Cargo where when an entire project
directory is renamed (preserving the target directory) then path
dependencies with build scripts would have their build scripts rereun
when building again. The problem with this was that when a build script
doesn't print `rerun-if-changed` Cargo's conservative fingerprint listed
an absolute path in it, which was intended to be a relative path.
The fix here is to use a relative path in the fingerprint to ensure that
it's not the reason a rebuild happens when directories are renamed.
This commit fixes an issue in Cargo where when an entire project
directory is renamed (preserving the target directory) then path
dependencies with build scripts would have their build scripts rereun
when building again. The problem with this was that when a build script
doesn't print `rerun-if-changed` Cargo's conservative fingerprint listed
an absolute path in it, which was intended to be a relative path.
The fix here is to use a relative path in the fingerprint to ensure that
it's not the reason a rebuild happens when directories are renamed.
Add a `-Zbuild-std-features` flag
This flag is intended to pair with `-Zbuild-std` as necessary to
configure the features that libstd is built with. This is highly
unlikely to ever be stabilized in any form (unlike `-Zbuild-std` which
we'd like to stabilize at some point), but can be useful for
experimenting with the standard library. For example today it can be
used to test changes to binary size by disabling backtraces.
My intention is that we won't need a `--no-default-features` equivalent
for libstd, where after rust-lang/rust#74377 is merged we can
unconditionally specify default features are disabled but the default
set of features lists `default`. That way if users want to override the
list *and* include the default feature, they can just be sure to include
`default`.
This flag is intended to pair with `-Zbuild-std` as necessary to
configure the features that libstd is built with. This is highly
unlikely to ever be stabilized in any form (unlike `-Zbuild-std` which
we'd like to stabilize at some point), but can be useful for
experimenting with the standard library. For example today it can be
used to test changes to binary size by disabling backtraces.
My intention is that we won't need a `--no-default-features` equivalent
for libstd, where after rust-lang/rust#74377 is merged we can
unconditionally specify default features are disabled but the default
set of features lists `default`. That way if users want to override the
list *and* include the default feature, they can just be sure to include
`default`.
clippy cleanups
Fixes a couple of clippy warnings.
Ignores clippy::collapsible_if warnings in the future (iirc there were not desired)
clippy::redundant_clone is enabled by default by clippy already.
Fix self-publish script.
Removes the dry run, since it won't work (verification fails since dependencies aren't actually published).
Also adds a sleep in-between publishing to give the index a moment to update.
Ensure `unstable.build-std` works like `-Zbuild-std`
This fixes an issue where the deserializer for `-Zbuild-std` was a bit
fancier than the `unstable.build-std` directive.
cc #8393
Make `cargo metadata` output deterministic
Uses BTreeMap instead of HashMap for the `cargo metadata` command, ensuring the output is sorted.
The change did not cause a measurable performance impact for running `cargo metadata` on `cargo` itself.
Fixes#8477
Uses BTreeMap instead of HashMap for the `cargo metadata` command.
The change did not cause a measurable performance impact for
running `cargo metadata` on `cargo` itself.
Fixes#8477
Switch to github actions
This commit switches our CI from Azure Pipelines to GitHub Actions. The intention here is to follow the "idiomatic" provider of CI for rust-lang, and otherwise GitHub Actions is better integrated with GitHub's UI right now too.
I'll need to tweak bors to actually `@bors: r+` this to have it successfully get merged, but I think it'd be good to get some review first.
Allow configuring unstable flags via config file
# Summary
This fixes#8127 by mapping the `unstable` key in `.cargo/config` to Z flags.
It should have no impact on stable/beta cargo, and on nightlies it gives folks the ability to configure Z flags for an entire project or workspace. This is meant to make it easier to try things like the [new features resolver](https://github.com/rust-lang/cargo/issues/8088) behavior, mtime-on-use, build-std, or timing reports across a whole project.
I've also included a small (but entirely independent) ergonomics change -- treating dashes and underscores identically in Z flags. That's along for the ride in this PR as the last commit, and is included here because it makes for more idiomatic toml file keys (`print_im_a_teapot = yes` vs `print-im-a-teapot = yes`). Please let me know if y'all would prefer that be in a separate PR, or not happen at all.
# Test Plan
Apologies if I've missed anything -- this is my first cargo contrib and I've tried to hew to the contributing guide. If I've slipped up, please let me know how and I'll fix it.
NB. My linux machine doesn't have multilib set up, so I disabled cross tests.
* `cargo test` passes for each commit in the stack
* I formatted each commit in the stack with `rustfmt`
* New tests are included alongside the relevant change for each change
* I've validated each test by locally undoing the code change they support and confirming failure.
* The CLI wins, for both enable and disabling Z flags, as you'd expect.
Keys in `unstable` which do not correspond with a Z flag will trigger an error indicating the invalid flag came from a config file read:
```
Invalid [unstable] entry in Cargo config
Caused by:
unknown `-Z` flag specified: an-invalid-flag
```
If you'd like to see a test case which isn't represented here, I'm happy to add it. Just let me know.
# Documentation
I've included commits in this stack updating the only docs page that seemed relevant to me, skimming the book -- `unstable.md`.
These tests demonstrate the current failure mode around
overlapping env keys and inner structs. To some extent this
is a limitation of mapping on to environment variables with
an underscore as both the namespace separator and the
substitute for dashes in flag names in that the mapping is
not strictly one-to-one.
Add support for rustc's `-Z terminal-width`.
This PR continues the work started in #7315, adding support for rustc's `-Z terminal-width` flag, which is used to trim diagnostic output to fit within the current terminal and was added in rust-lang/rust#63402 (with JSON emitter support in rust-lang/rust#73763).
At the time of writing, rust-lang/rust#73763 isn't in nightly, so the test added in this PR will fail, but it should pass tomorrow (I've confirmed that it works with a local rustc build).
cc @estebank
Avoid colliding with older Cargo fingerprint changes
The fingerprint format Cargo stores changed recently in a way that
older Cargos cannot understand. Unfortunately though older Cargos are
colliding on some compilation units trying to read the new format and
they're bailing out. This commit fixes this issue by ensuring that the
location for fingerprint metadata is different in older Cargos and newer
Cargos.
Fingerprint metadata is always stored in a location with a hash in the
file name. This hash typically includes the hash of rustc's version
information itself, but for units which don't have a `Metadata` it's a
much simpler hash which is much more likely to collide with other
versions of Cargo. The fix in this commit is to extract the metadata
version that we're hashing to a constant, and then also hash it for
generating a filesystem location to house fingerprint data for a unit
that has no `Metadata`.
Closes#8472Closes#8298
The fingerprint format Cargo stores changed recently in a way that
older Cargos cannot understand. Unfortunately though older Cargos are
colliding on some compilation units trying to read the new format and
they're bailing out. This commit fixes this issue by ensuring that the
location for fingerprint metadata is different in older Cargos and newer
Cargos.
Fingerprint metadata is always stored in a location with a hash in the
file name. This hash typically includes the hash of rustc's version
information itself, but for units which don't have a `Metadata` it's a
much simpler hash which is much more likely to collide with other
versions of Cargo. The fix in this commit is to extract the metadata
version that we're hashing to a constant, and then also hash it for
generating a filesystem location to house fingerprint data for a unit
that has no `Metadata`.
Closes#8472
This patch changes how ConfigMapAccess iterates k/v pairs when
deserializing structs.
Previously we produced keys for exactly the set of fields needed
for a struct, and errored out in the deserializer if we can't find
anything for that field.
This patch makes us produces keys from the union of two sets:
1. All fields that are both needed for the struct and can be found
in the environment.
2. All fields in the config table.
This change allows serde's codegen to handle both missing and
unknown fields via the usual derive annotations (default or
deny_unknown_fields respectively)
Disable long_file_names test if not supported on Windows.
This test will fail on Windows if Cargo's target directory is over 80 characters long (as it is in rust's CI).
This commit modifies the parsing of `-Z terminal-width` so that it can
optionally take a value and only uses `accurate_err_width` when the user
does not provide a value - therefore making the emission of `-Z
terminal-width` opt-in.
Signed-off-by: David Wood <david@davidtw.co>
This commit adds support for rustc's `-Z terminal-width` flag, which is
used to trim diagnostic output to fit within the current terminal.
Co-authored-by: David Wood <david@davidtw.co>
Signed-off-by: David Wood <david@davidtw.co>
The current behavior, with the default-false workaround in place,
is not able to identify extra members of the `unstable` table, so
it can't error on unexpected members.
I'll add this test back in with a Deserializer refactor to clean up
the extra logic added to CliUnstable.
Tests are currently failing, looks like there's something in the
Deserializer impl that's forcing field-missing errors even when
the serde `default` annotation is applied.
