This commit continues the work from #9112 to enable `unpacked` split
debuginfo on macOS targets by default. This has been discussed on [internals]
for awhile now and no breakage has emerged while significant speedups
have. This is expected to be a compile-time and `target`-directory size
win for almost all macOS Rust projects.
While breakage is possible it's possible to mitigate this with
project-local or global cargo configuration of the `dev` and `test` profiles.
[internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
Split out cargo-util package for cargo-test-support.
This splits out code from `cargo` that was being used by `cargo-test-support` in order to remove the dev-dependency cycle on the `cargo` crate. The intent here is to improve development build times. On my system, `touch src/cargo/lib.rs ; cargo check --profile=test` goes from 6.4 seconds to 3.5. I suspect more substantial changes (more than just `touch`) should exhibit even more improvements.
This has a substantial downside that it is another package to manage publishing, particularly with making sure the version is updated correctly for crates.io. I'm uncertain if on balance it is worth it, but I lean towards moving forward with it.
cargo-test-support wasn't using any of the caching or other logic from
Rustc, so this just swaps with a very basic implementation in order to
remove the dependency on `cargo`.
Forbid setting `RUSTC_BOOTSTRAP` from a build script on stable
Instead, recommend `RUSTC_BOOTSTRAP=crate_name`. If cargo is using a nightly toolchain, or if RUSTC_BOOTSTRAP was set in *cargo*'s build environment, the error is downgraded to a warning, since the variable won't affect the build.
This is mostly the same as suggested in https://github.com/rust-lang/cargo/issues/7088#issuecomment-713867773, except that `RUSTC_BOOTSTRAP=` values other than 1 are treated the same as `RUSTC_BOOTSTRAP=1`. My reasoning was that https://github.com/rust-lang/rust/pull/77802 is now on 1.50 stable, so some crates may have started using it, and I would still prefer not to give hard errors when there's no workaround.
Closes https://github.com/rust-lang/cargo/issues/7088.
r? `@joshtriplett`
testsuite: Use split debuginfo on macos.
This switches the testsuite to use "unpacked" debuginfo on macos, which is a substantial performance boost. On my system, the testsuite runs 1.55 times faster with this change. Along with #9206, total testsuite time is 3.1 times faster.
`nightly_features_allowed()` is no longer doing any work, so it can be
accessed directly. This also renames the `enable_nightly_features` field
to `nightly_features_allowed`.
Previously, since `ENABLE_NIGHTLY_FEATURES` and
`NIGHTLY_FEATURES_ENABLED` were thread locals, reading them in any other
thread would always say nightly features were disabled. Now, they are
tied to the `Context` itself, so it is both more clear how the variables
are being set and fixes the behavior within work threads.
Note that `Context` is not thread-safe, so this passes a boolean through
to `BuildOutput::parse`.
Updates to edition handling.
This introduces some updates for edition handling (split into commits for review). In short:
* `cargo-features = ["edition2021"]` can be used to opt-in to 2021 support without needing to pass around `-Z unstable-options`. I tried to emphasize in the docs that this is only for testing and experimentation.
* Make `"2"` the default resolver for 2021 edition.
* Make `cargo fix --edition` mean the "next" edition from the present one, and support 2021. Also, if already at the latest edition, generate a warning instead an error.
* I decided to allow `cargo fix --edition` from 2018 to 2021 on the nightly channel without an explicit opt-in. It's tricky to implement with an opt-in (see comment in diff).
Partial for #9048.
Fixes#9047.
What was previously "Fixing" was a message for after the fixes had
been applied. I think it would be clearer if it said "Fixed",
to indicate that the fixes had actually finished.
The new "Fixing" is posted just before it starts. This is verbose-only
since it is a little noisy.
Add split-debuginfo profile option
This commit adds a new `split-debuginfo` option to Cargo compilation
profiles which gets forwarded to the `-Csplit-debuginfo` codegen option
in rustc. This commit also sets the default, only on macOS, to be
`-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage
rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds
while also preserving a pleasant debugging experience by default. This
should lead to much faster incremental build times on macOS since
`dsymutil` isn't exactly the speediest tool in the world.
This is technically a breaking change in Cargo because we're no longer
by-default producing the `*.dSYM` folders on macOS. If those are still
desired, however, authors can always run `dsymutil` themselves or
otherwise configure `split-debuginfo = 'packed'` in their
manifest/profile configuration.
This commit updates the rustc info cache to cache failures to execute
rustc as well as successes. This fixes a weird issue where if you're
probing for flags the `rustc_info_cache` test fails on channels which
don't have the flag since previously a failure to execute rustc resulted
in never caching the result.
This commit adds a new `split-debuginfo` option to Cargo compilation
profiles which gets forwarded to the `-Csplit-debuginfo` codegen option
in rustc. This commit also sets the default, only on macOS, to be
`-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage
rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds
while also preserving a pleasant debugging experience by default. This
should lead to much faster incremental build times on macOS since
`dsymutil` isn't exactly the speediest tool in the world.
This is technically a breaking change in Cargo because we're no longer
by-default producing the `*.dSYM` folders on macOS. If those are still
desired, however, authors can always run `dsymutil` themselves or
otherwise configure `split-debuginfo = 'packed'` in their
manifest/profile configuration.
Implement external credential process. (RFC 2730)
This adds a config setting for an external process to run to fetch the token for a registry. See `unstable.md` for more details.
As part of this, it adds a new `logout` command. This is currently gated on nightly with the appropriate `-Z` flag.
I have included four sample wrappers that integrate with the macOS Keychain, Windows Credential Manager, GNOME libsecret, and 1password. I'm not sure if we'll ultimately ship these, but I would like to. Primarily this provided a proof-of-concept to see if the design works.
**Patch Walkthrough**
This is a brief overview of the changes:
- Adds the `logout` command. With `cargo logout -Z unstable-options`, this allows removing the `token` from `.cargo/credentials`. With `cargo logout -Z credential-process`, this launches the process with the `erase` argument to remove the token from storage.
- Credential-process handling is in the `ops/registry/auth.rs` module. I think it is pretty straightforward, it just launches the process with the appropriate store/get/erase argument.
- `ops::registry::registry()` now returns the `RegistryConfig` to make it easier to pass the config information around.
- `crates/credential/cargo-credential` is a helper crate for writing credential processes.
- A special shorthand of the `cargo:` prefix for a credential process will launch the named process from the `libexec` directory in the sysroot (or, more specifically, the `libexec` directory next to the `cargo` process). For example `credential-process = "cargo:macos-keychain"`. My intent is to bundle these in the pre-built rust-lang distributions, and this should "just work" when used with rustup. I'm not sure how that will work with other Rust distributions, but I'm guessing they can figure it out. This should make it much easier for users to get started, but does add some integration complexity.
**Questions**
- I'm on the fence about the name `credential-process` vs `credentials-process`, which sounds more natural? (Or something else?)
- I'm uneasy about the behavior when both `token` and `credential-process` is specified (see `warn_both_token_and_process` test). Currently it issues a warning and uses `token`. Does that make sense? What about the case where you have `registries.foo.token` for a specific registry, but then have a general `registry.credential-process` for the default (it currently warns and uses the token, maybe it should not warn?)?
- I am still pretty uneasy with writing FFI wrappers, so maybe those could get a little extra scrutiny? They seem to work, but I have not extensively tested them (I tried login, publish, and logout). I have not previously used these APIs, so I am not familiar with them.
- Testing the wrappers I think will be quite difficult, because some require TTY interaction (and 1password requires an online account). Or, for example in the macOS case, it has GUI dialog box where I can use my fingerprint scanner. Right now, I just build them in CI to make sure they compile.
- 1password is a little weird in that it passes the token on the command-line, which is not very secure on some systems (other processes can see these sometimes). The only alternative I can think of is to not support `cargo login` and require the user to manually enter the token in the 1password GUI. I don't think the concern is too large (1password themselves seem to think it is acceptable). Should this be OK?
- I'm a little uneasy with the design of `cargo login`, where it passes the token in stdin. If the wrapper requires stdin for user interaction (such as entering a password to unlock), this is quite awkward. There is a hack in the 1password example that demonstrates using `/dev/tty` and `CONIN$`, which *seems* to work, but I'm worried is fragile. I'm not very comfortable passing the token in environment variables, because those can be visible to other processes (like CLI args), but in some situations that shouldn't be too risky. Another option is to use a separate file descriptor/handle to pass the token in. Implementing that in Rust in a cross-platform way is not terribly easy, so I wanted to open this up for discussion first.
