When running `cargo package` the verify the build cache
(target-dir/build-dir) will not be used and all dependencies will be
recompiled.
This is inconsistent as setting target dir (via `CARGO_TARGET_DIR` for
example) will cause `cargo package` to reuse the build cache.
This commit changes the default behavior to always use the build cache,
matching the behavior of having target-dir set.
A user will now be able to use flags like `--workspace` with `cargo
publish`.
`cargo package` will now also work with those flags without having to
pass `--no-verify --exclude-lockfile`.
Many release tools have come out that solve this problem.
They will still need a lot of the logic that went into that for other
parts of the release process.
However, a cargo-native solution allows for:
- Verification during dry-run
- Better strategies for waiting for the publish timeout
`cargo publish` is non-atomic at this time.
If there is a server side error, network error, or rate limit during the publish,
the workspace will be left in a partially published state.
Verification is done before any publishing so that won't affect things.
There are multiple strategies we can employ for improving this over time,
including
- atomic publish
- `--idempotent` (#13397)
- leave this to release tools to manage
This includes support for `--dry-run` verification.
As release tools didn't have a way to do this before,
users may be surprised at how slow this is because a `cargo build` is
done instead of a `cargo check`. This is being tracked in #14941.
This adds to `cargo package` the `--registry` and `--index` flags to
help with resolving dependencies when depending on a package being
packaged at that moment.
These flags are only needed when a `cargo package --workspace` operation
would have failed before due to inability to find a locally created
dependency.
Regarding the publish timeout, `cargo publish --workspace` publishes
packages in batches and we only timeout if nothing in the batch has
finished being published within the timeout, deferring the rest to the
next wait-for-publish. So for example, if you have packages `a`, `b`, `c` then
we'll wait up to 60 seconds and if only `a` and `b` were ready in that time,
we'll then wait another 60 seconds for `c`.
During testing, users ran into issues with `.crate` checksums that we've
not been able to reproduce since:
- https://github.com/rust-lang/cargo/issues/1169#issuecomment-2567995987
- #14396
By stabilizing this, Cargo's behavior becomes dependent on an overlay
registry.
When generating a lockfile or verifying a package, we overlay the
locally generated `.crate` files on top of the registry so the registry
appears as it would and everything works.
If there is a conflict with a version, the local version wins which is
important for the dry-run mode of release tools as they won't have
bumped the version yet.
Our concern for the overlay registry is dependency confusion attacks.
Considering this is not accessible for general user operations, this
should be fine.
Fixes#1169Fixes#10948
### What does this PR try to resolve?
This PR reworks `cargo-test-support` and `testsuite` to use Snapbox's
[`cargo_bin!()`](https://docs.rs/snapbox/latest/snapbox/cmd/macro.cargo_bin.html)
instead of
[`cargo_bin()`](https://docs.rs/snapbox/latest/snapbox/cmd/fn.cargo_bin.html)
which makes assumptions about the structure of Cargo's build directory.
`cargo_bin!()` uses `CARGO_BIN_EXE_*` for locating the `cargo` binary
which should be more resilient to directory/layout changes.
Linking a relevant Zulip discussion
[here](https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/cargo_bin_exe.20and.20tests/with/513638220)[#t-cargo
> cargo_bin_exe and
tests](https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/cargo_bin_exe.20and.20tests/with/513638220)
As shown in that link, we could make these variables available at
runtime and not need to do this. However, `cargo-test-support`, as an
API, is a bit weird in that it is baking in support for one specific
binary. This can be confusing for callers and makes it more annoying for
callers provide their own `fn cargo`, e.g. see crate-ci/cargo-fixit#7
### Implementation Notes
`cargo_bin!()` only works when being called from the `testsuite` as it's
only set when executing integration tests and `cargo-test-support` is a
regular crate.
To make this change, I introduced an extension trait `CargoProjectExt`
in `testsuite` for running `.cargo()` and implemented it on `Project`.
In `cargo-test-support` other functionality relies on `.cargo()` so
these also needed to be moved to `testsuite`
*
[`src/tools.rs`](https://github.com/rust-lang/cargo/blob/master/crates/cargo-test-support/src/tools.rs)
* Parts
[`src/cross_compile`](https://github.com/rust-lang/cargo/blob/master/crates/cargo-test-support/src/cross_compile.rs)
* I had to split this up unfortunately, as `disabled()` requires running
Cargo to check if we should disable cross compile tests.
* Other fns in `cross_compile` are used in `cargo-test-support` so
moving everything to `testsuite` would have ended up requiring moving
many things to test suite.
### How to test and review this PR?
I'd definitely recommend reviewing commit by commit.
There are a lot of diffs due to the nature of reorganizing things.
I did my best to split things things into smaller PRs but they still
contain a lot of `use` statement diffs.
r? @epage
This doesn't need to be an hard error because missing
publish for the entire worspace should be a fairly visible.
However, this is open for future to configure via Cargo lint system.
This changes how cargo-publish works with the unstable
`-Zpackage-workspace` feature.
Before this, when publishing the entire workspace,
like `cargo publish --workspace`,
if there is a package with `package.pulibsh=false,
it'll fail the entire publish process.
After this, when `--workspace` is passed,
or when publishing the virtual workspace,
the intent is more like “publish all publishable in this workspace”,
so skip `publish=false` packages and proceed to publish others.
The new overall behavior looks like this:
- `cargo publish` (inside a `package.publish = false` package): error
- `cargo publish -p publishable -p unpublishable`: error
- `cargo publish --workspace`: skips `package.publish = false
See https://github.com/rust-lang/cargo/issues/15006#issuecomment-2847660911
I didn't extend this to multiple packages restricted to specific
registries.
It seems less likely to be a problem and more complex to gather and
report.
This was inspired by feedback left at #10948
I found a bug in the manifest parser and figured this would help make it
more obvious.
Since I was already changing the order, I figure I'm make things a
little more logical (user-facing first, implementtion details later)
Originally it was only included for packages that have executables or
examples for `cargo install`, however this causes inconsistencies and
is kind of unexpected nowadays, e.g. with cdylib crates.
Including it always only slightly increases the crate size and allows
for all crates to know a set of dependency versions that were working,
which can make regression tracking easier.
Fixes https://github.com/rust-lang/cargo/issues/13447
fix(publish): Downgrade version-exists error to warning on dry-run
### What does this PR try to resolve?
Especially for beta, this was the most conservative, minimal change.
Fixes#14721
### How should we test and review this PR?
### Additional information
This will get cherry-picked to beta
`Registry::describe_source` will be used as default error message, but if the message
come from `Registry::describe_source` is empty, then it uses `SourceId` message
Due to problems we ran into with #14016, we're removing implicit
features from the 2024 edition to give ourselves more time to design it
as we should.
I could have added a new flag for this or made an EditionNext but I
decided to remove it in the hopes to avoid any path dependency in
solving this the next time.
PR #5335 added `autobins`, etc for #5330. Nowhere in there is
discussion of `autolib`.
Cargo script disables support for additional build-targets by disabling
discovery.
Except we don't have a way to disable discovery of `autolib`, leading to #14476.
By adding `autolib`, we can continue in that direction.
This also allows us to bypass inferring of libs on published packages,
like all other build-targets which were handled in #13849.
As this seems fairly low controversy, this insta-stabilizes the field.
In prior versions of Cargo, users will get an "unused manifest key"
warning.
For packags where this is set by `cargo publish`, the warning will be suppressed and things will work as normal.
For `cargo vendor`, the same except there will be some churn in the
vendored source as this field will now be set.
For local development, it should be rare to set `autolib` so the lack of
error by discovering a file when this is set shouldn't be a problem.
Fixes#14476
This commit makes lockfile version 4 the default version when Cargo
tries to write to a lockfile.
The lockfile version 4 has been stabilized since 1.78.0,
and will become default in 1.83.0.
the length of transition period is pretty similar as before.
One caveat is that in other output from Cargo,
e.g., `cargo metatada`, status messages,
`SourceID` will display in the v4 URL encoded format.
This shouldn't affect the majority of Rust users,
as `SourceId` representation should be opaque to them,
unless comparing `SourceId` across different version of toolchains.
First, we added support for stripping of local-only dev-dependencies.
This was dual-implemented for `Cargo.toml` and `Summary`.
This left off stripping of `dep/feature` that reference dev-dependencies
(enabling features within dev-dependencies).
When we fixed this, we again dual-implemented it.
The `Cargo.toml` version was correct but the `Summary` version was
instead stripping too many features,
particularly features that reference renamed dependencies.
We didn't have tests for this case and it wasn't caught earlier because
crates.io re-generates the `Summary` from `Cargo.toml`, ignoring what we
post.
That makes this only show up with custom registries that trust what
Cargo posts.
Rather than fixing the `Summary` generation, I remove the
dual-implementation and instead generate the `Summary` from the
published `Cargo.toml`.
Unfortunately, we don't have access directly to the packaged
`Cargo.toml`.
It could be passed around and I originally did so, hoping to remove use
of the local `Package`.
However, the local `Package` is needed for things like reading the
`README`.
So I scaled back and isolate the change to only what needs it.
This also makes it easier for `prepare_transmit` callers.
Fully open to someone exploring removing this extra `prepare_for_publish` in the future.
Fixes#14321
We now include the prelude in so many places, this simplifies how we can
present how `cargo-test-support` works.
Yes, this included some `use` clean ups but its already painful enough
walking through every test file, I didn't want to do it twice.
