A user will now be able to use flags like `--workspace` with `cargo
publish`.
`cargo package` will now also work with those flags without having to
pass `--no-verify --exclude-lockfile`.
Many release tools have come out that solve this problem.
They will still need a lot of the logic that went into that for other
parts of the release process.
However, a cargo-native solution allows for:
- Verification during dry-run
- Better strategies for waiting for the publish timeout
`cargo publish` is non-atomic at this time.
If there is a server side error, network error, or rate limit during the publish,
the workspace will be left in a partially published state.
Verification is done before any publishing so that won't affect things.
There are multiple strategies we can employ for improving this over time,
including
- atomic publish
- `--idempotent` (#13397)
- leave this to release tools to manage
This includes support for `--dry-run` verification.
As release tools didn't have a way to do this before,
users may be surprised at how slow this is because a `cargo build` is
done instead of a `cargo check`. This is being tracked in #14941.
This adds to `cargo package` the `--registry` and `--index` flags to
help with resolving dependencies when depending on a package being
packaged at that moment.
These flags are only needed when a `cargo package --workspace` operation
would have failed before due to inability to find a locally created
dependency.
Regarding the publish timeout, `cargo publish --workspace` publishes
packages in batches and we only timeout if nothing in the batch has
finished being published within the timeout, deferring the rest to the
next wait-for-publish. So for example, if you have packages `a`, `b`, `c` then
we'll wait up to 60 seconds and if only `a` and `b` were ready in that time,
we'll then wait another 60 seconds for `c`.
During testing, users ran into issues with `.crate` checksums that we've
not been able to reproduce since:
- https://github.com/rust-lang/cargo/issues/1169#issuecomment-2567995987
- #14396
By stabilizing this, Cargo's behavior becomes dependent on an overlay
registry.
When generating a lockfile or verifying a package, we overlay the
locally generated `.crate` files on top of the registry so the registry
appears as it would and everything works.
If there is a conflict with a version, the local version wins which is
important for the dry-run mode of release tools as they won't have
bumped the version yet.
Our concern for the overlay registry is dependency confusion attacks.
Considering this is not accessible for general user operations, this
should be fine.
Fixes#1169Fixes#10948
### What does this PR try to resolve?
This is prep for updating `toml` which will change some of these error
messages
### How to test and review this PR?
Fixes#15647.
When dry-run publishing workspace without bumping versions first, the
package-verification step would fail because it would see checksum
mismatches between the old lock file (that saw index deps) and the new
lock file where those index deps got replaced by local packages with the
same version.
In this PR, the packaging step modifies the old lock file's checksums
before re-resolving, but only in dry-run mode.
### What does this PR try to resolve?
This PR reworks `cargo-test-support` and `testsuite` to use Snapbox's
[`cargo_bin!()`](https://docs.rs/snapbox/latest/snapbox/cmd/macro.cargo_bin.html)
instead of
[`cargo_bin()`](https://docs.rs/snapbox/latest/snapbox/cmd/fn.cargo_bin.html)
which makes assumptions about the structure of Cargo's build directory.
`cargo_bin!()` uses `CARGO_BIN_EXE_*` for locating the `cargo` binary
which should be more resilient to directory/layout changes.
Linking a relevant Zulip discussion
[here](https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/cargo_bin_exe.20and.20tests/with/513638220)[#t-cargo
> cargo_bin_exe and
tests](https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/cargo_bin_exe.20and.20tests/with/513638220)
As shown in that link, we could make these variables available at
runtime and not need to do this. However, `cargo-test-support`, as an
API, is a bit weird in that it is baking in support for one specific
binary. This can be confusing for callers and makes it more annoying for
callers provide their own `fn cargo`, e.g. see crate-ci/cargo-fixit#7
### Implementation Notes
`cargo_bin!()` only works when being called from the `testsuite` as it's
only set when executing integration tests and `cargo-test-support` is a
regular crate.
To make this change, I introduced an extension trait `CargoProjectExt`
in `testsuite` for running `.cargo()` and implemented it on `Project`.
In `cargo-test-support` other functionality relies on `.cargo()` so
these also needed to be moved to `testsuite`
*
[`src/tools.rs`](https://github.com/rust-lang/cargo/blob/master/crates/cargo-test-support/src/tools.rs)
* Parts
[`src/cross_compile`](https://github.com/rust-lang/cargo/blob/master/crates/cargo-test-support/src/cross_compile.rs)
* I had to split this up unfortunately, as `disabled()` requires running
Cargo to check if we should disable cross compile tests.
* Other fns in `cross_compile` are used in `cargo-test-support` so
moving everything to `testsuite` would have ended up requiring moving
many things to test suite.
### How to test and review this PR?
I'd definitely recommend reviewing commit by commit.
There are a lot of diffs due to the nature of reorganizing things.
I did my best to split things things into smaller PRs but they still
contain a lot of `use` statement diffs.
r? @epage
Two tests were using `.rust_version("1.90.0")` for a "newer" Rust, but
this failed in rust-lang/rust#142792 for the start of the *real* 1.90.0.
Let's bump that to 1.999.0 so we won't have to deal with it for a while.
The rustc `-Zhint-mostly-unused` flag tells rustc that most of a crate
will go unused. This is useful for speeding up compilation of large
dependencies from which you only use a few items. Plumb that option
through as a profile option, to allow specifying it for specific
dependencies:
```toml
[profile.dev.package.huge-mostly-unused-dependency]
hint-mostly-unused = true
```
To enable this feature, pass `-Zprofile-hint-mostly-unused`. However,
since this option is a hint, using it without passing
`-Zprofile-hint-mostly-unused` will only warn and ignore the profile
option. Versions of Cargo prior to the introduction of this feature will
give an "unused manifest key" warning, but will otherwise function
without erroring. This allows using the hint in a crate's `Cargo.toml`
without mandating the use of a newer Cargo to build it.
Add a test verifying that the profile option gets ignored with a warning
without passing `-Zprofile-hint-mostly-unused`, and another test
verifying that it gets handled when passing
`-Zprofile-hint-mostly-unused`.
### What does this PR try to resolve?
As of #15625, the manifest path argument in `cargo clippy
--manifest-path foo/Cargo.toml --fix` will be ignored. All the workspace
members will be built. The cause is due to the `reload` usage in
`cargo::ops::fix`. We reload the `root_manifest` in the function, which
contains all workspace members.
Will close#15625.
### How to test and review this PR?
The first commit in the PR demonstrates the current problem, and the
second commit corrects it. Use `cargo test --test testsuite
workspaces::fix_only_check_manifest_path_member` to see the test
results.
