### What does this PR try to resolve?
Found this as I was preparing for stabilization.
As we create a graph of packages to ensure publish order, a self-cycle
shouldn't matter, so we can skip tracking it.
### How to test and review this PR?
Remap all paths pointing to `build.build-dir`,
i.e., `[BUILD_DIR]/debug/deps/foo-[HASH].dwo` would be remapped to
`/cargo/build-dir/debug/deps/foo-[HASH].dwo`
(note the `/cargo/build-dir` prefix).
This covers scenarios like:
* Build script generated code. For example, a build script may call `file!`
macros, and the associated crate uses [`include!`] to include the expanded
[`file!`] macro in-place via the `OUT_DIR` environment.
* On Linux, `DW_AT_GNU_dwo_name` that contains paths to split debuginfo
files (dwp and dwo).
### What does this PR try to resolve?
Before this some end-to-end tests didn't cover Windows platforms.
After this, we cover windows-msvc for
* End-to-end debugger tests.
* Check path is trimmed with symbol viewers like `strings`.
windows-gnu isn't covered
### How to test and review this PR?
There are things needing attentions:
* This adds a new CI job for window-msvc "nightly" toolchain.
* In 2f923b3ff25f847d we don't check if an executable's availability by
running `<cmd> --version`. Instead, we check the file execute bit.
* Enabled windows-msvc tests rely on the software provided by [GitHub
windows runner
image](e330e24b7e/images/windows/Windows2022-Readme.md)
* Windows SDK which provides cdb and other debugger tools
* `strings` is provided by MinGW.
GitHub Actions has Windows SDK pre-installed,
so cdb should be available at that path always.
Additionally, this adds a new CI job for windows-msvc nightly toolchain
### What does this PR try to resolve?
This PR fixes a bug where `cargo add` breaks symlinks to Cargo.toml
files. Currently, when Cargo.toml is a symlink and `cargo add` is used
to add a dependency, the symlink is replaced with a regular file,
breaking the link to the original target file.
This issue was reported in #15241 where a user who relies on symlinked
Cargo.toml files found that `cargo add` breaks their workflow.
Fixes#15241
### How should we test and review this PR?
I've modified `LocalManifest::write()` to check if the path is a
symlink, and if so, follow it to get the actual target path. This
ensures we write to the actual file rather than replacing the symlink.
I've also added a test in `tests/testsuite/cargo_add/symlink.rs` that:
1. Creates a symlinked Cargo.toml file
2. Runs `cargo add` to add a dependency
3. Verifies the symlink is preserved and the dependency is added to the
target file
I've manually tested this fix and confirmed it works correctly.
- Preserve symlinks when writing files atomically in write_atomic()
- Update test to verify correct symlink preservation behavior
- Apply rustfmt formatting
This fixes the issue where cargo add would replace symlinked Cargo.toml
files with regular files, breaking the symlink to the original target.
Fixes#15241
`cargo test` will implicitly build examples as examples binaries
(without --test) by default, when no target selection flags provided.
We don't have test exercising this, so add one.
This test shows the current behavior where cargo add replaces
symlinked Cargo.toml files with regular files. The test passes,
documenting this problematic behavior.
This separates the concern of two different "mode".
- UserIntent: focus on the overall goal of the build
- CompileMode: the actual compile operation for each unit
This is a preparation of adding `-Zno-link`/`-Zlink-only` support,
which we'll have `CompileMode::Link` but that doesn't make sense to
show up in `UserIntent`.
This adds support for the "future" edition which was added to rustc in
https://github.com/rust-lang/rust/pull/137606.
To enable support for unstable editions, this introduces a new
`unstable-editions` cargo feature. The intent is that instead of having
a new feature for each edition that we reuse this feature for all new
editions. I don't see a particular reason we should have a separate one
for each edition, and this helps a bit with scalability and simplifies
some of the edition process.
This also includes a change to rework `supports_compat_lint` explained
in the comment.
### What does this PR try to resolve?
With rust-lang/rust#140035 now merged, we can rely on that rather than
dirty hacks
This is part of #12207
### How should we test and review this PR?
### Additional information
This is meant to fixes#13191
As git sources and registry sources are considered immutable.
I don't think there is any reason excluding those files.
There might be a little chance local Git repositories might have those,
though that is a rare use case.
Alternatively,
we could reject all `.rej`/`.orig` files but `Cargo.toml.orig`.
### What does this PR try to resolve?
`PathSource::list_files` has some heurstic rules for listing files.
Those rules are mainly designed for `cargo package`.
Previously, cargo-vendor relies on those rules to understand what files
to vendor. However, it shouldn't use those rules because:
* Package extracted from a `.crate` tarball isn't Git-controlled, some
rules may apply differently.
* The extracted package already went through `PathSource::list_files`
during packaging. It should be clean enough.
* Should keep crate sources from registry sources in a pristine state,
which is exactly what vendoring is meant for.
Instead, we switch to direct extraction into the vendor directory
to ensure source code is the same as in the `.crate` tarball.
### How should we test and review this PR?
There is already a test `vendor::package_exclude` for the fix of #9054,
though now I think it is not a good fix. The test change shows the
correct behavior change.
I am not sure if we want more tests.
Also, there are some caveats with this fix:
* The overwrite protection in `unpack_package` assumes the unpack
directory is always `<pkg>-<version`>.
We don't want to remove this,
but for cargo-vendor supports vendoring without version suffix.
For that case, we need a temporary staging area,
and move the unpacked source then.
* The heurstic in `PathSource::list_files` did something "good" in
general cases, like excluding hidden directories. That means
common directories like `.github` or `.config` won't be vendored.
After this, those get included. This is another round of churns.
We might want to get other `cargo-vendor` changes along with this
in one single release.
### Additional information
* Fixes#9054
* Fixes#9555
* Fixes#9575
* Fixes#11000
* Fixes#14034
* Fixes#15080
* Fixes#15090
This also opens a door for
* #10310
* #13191
Since we are changing vendored source (again), we might want to remove
the `.rej`/`.orig` special rules in cargo-vendor, as well as look into
the new source-dedup vendor dir layout.
<!-- TRIAGEBOT_START -->
<!-- TRIAGEBOT_SUMMARY_START -->
### Summary Notes
-
[benchmark-result](https://github.com/rust-lang/cargo/pull/15514#issuecomment-2870275766)
by [weihanglo](https://github.com/weihanglo)
Generated by triagebot, see
[help](https://forge.rust-lang.org/triagebot/note.html) for how to add
more
<!--
TRIAGEBOT_SUMMARY_DATA_START$${"entries_by_url":{"https://github.com/rust-lang/cargo/pull/15514#issuecomment-2870275766":{"title":"benchmark-result","comment_url":"https://github.com/rust-lang/cargo/pull/15514#issuecomment-2870275766","author":"weihanglo"}}}$$TRIAGEBOT_SUMMARY_DATA_END
-->
<!-- TRIAGEBOT_SUMMARY_END -->
<!-- TRIAGEBOT_END -->
`PathSource::list_files` has some heurstic rules for listing files.
Those rules are mainly designed for `cargo package`.
Previously, cargo-vendor relies on those rules to understand what
files to vendor. However, it shouldn't use those rules because:
* Package extracted from a `.crate` tarball isn't Git-controlled,
some rules may apply differently.
* The extracted package already went through `PathSource::list_files`
during packaging. It should be clean enough.
* Should keep crate sources from registry sources in a pristine state,
which is exactly what vendoring is meant for.
Instead, we switch to direct extraction into the vendor directory
to ensure source code is the same as in the `.crate` tarball.
There are some caveats:
* The overwrite protection in `unpack_package` assumes the unpack
directory is always `<pkg>-<version`>.
We don't want to remove this,
but for cargo-vendor supports vendoring without version suffix.
For that case, we need a temporary staging area,
and move the unpacked source then.
* The heurstic in `PathSource::list_files` did something "good" in
general cases, like excluding hidden directories. That means
common directorys like `.github` or `.config` won't be vendored.
After this, those get included. This is another round of churns.
We might want to get other `cargo-vendor` changes along with this
in one single release.
### What does this PR try to resolve?
- As issue https://github.com/rust-lang/rust/issues/125246 has already
been fixed, there must be no need for commenting out `--sysroot` anymore
in the tests.
- The bug mentioned in `shared_std_dependency_rebuild()` looks to be
already solved and does not reproduce. So I guess it's safe to
un-comment those lines.
This doesn't need to be an hard error because missing
publish for the entire worspace should be a fairly visible.
However, this is open for future to configure via Cargo lint system.
This changes how cargo-publish works with the unstable
`-Zpackage-workspace` feature.
Before this, when publishing the entire workspace,
like `cargo publish --workspace`,
if there is a package with `package.pulibsh=false,
it'll fail the entire publish process.
After this, when `--workspace` is passed,
or when publishing the virtual workspace,
the intent is more like “publish all publishable in this workspace”,
so skip `publish=false` packages and proceed to publish others.
The new overall behavior looks like this:
- `cargo publish` (inside a `package.publish = false` package): error
- `cargo publish -p publishable -p unpublishable`: error
- `cargo publish --workspace`: skips `package.publish = false
See https://github.com/rust-lang/cargo/issues/15006#issuecomment-2847660911
### What does this PR try to resolve?
Cargo registries that return HTTP 429 when the service is overloaded
expect the client to retry the request automatically after a delay.
Cargo currently does not retry for HTTP 429.
### What changed?
* Adds HTTP 429 (too many requests) as a spurious HTTP error to enable
retries.
* Parse the
[Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Retry-After)
HTTP header to determine how long to wait before a retry.
In this implementation, the maximum delay is limited to Cargo's existing
limit of 10 seconds. We could consider increasing that limit for this
case, since the server is explicitly requesting the delay.
### What does this PR try to resolve?
Cargo script considers itself part of the workspace if the script is
located in a sub-directory of a workspace (presumably since
https://github.com/rust-lang/cargo/pull/15168). This becomes an issue
when using a custom registry that is defined in the `.cargo/config.toml`
within the workspace. Cargo script does not take that file into account
and fails with ``registry index was not found in any configuration:
`test-reg` ``.
### How should we test and review this PR?
This PR adds a regression test and makes cargo script ignore the
surrounding workspace.
The test ~~will fail without the fix in the second commit and~~ can be
used to reproduce the issue.
### Additional information
The issue started occurring with `nightly-2025-02-16`.
Related to https://github.com/rust-lang/cargo/issues/12207.
<!--
Thanks for submitting a pull request 🎉! Here are some tips for you:
* If this is your first contribution, read "Cargo Contribution Guide"
first:
https://doc.crates.io/contrib/
* Run `cargo fmt --all` to format your code changes.
* Small commits and pull requests are always preferable and easy to
review.
* If your idea is large and needs feedback from the community, read how:
https://doc.crates.io/contrib/process/#working-on-large-features
* Cargo takes care of compatibility. Read our design principles:
https://doc.crates.io/contrib/design.html
* When changing help text of cargo commands, follow the steps to
generate docs:
https://github.com/rust-lang/cargo/tree/master/src/doc#building-the-man-pages
* If your PR is not finished, set it as "draft" PR or add "WIP" in its
title.
* It's ok to use the CI resources to test your PR, but please don't
abuse them.
Explain the motivation behind this change.
A clear overview along with an in-depth explanation are helpful.
You can use `Fixes #<issue number>` to associate this PR to an existing
issue.
### How should we test and review this PR?
Demonstrate how you test this change and guide reviewers through your
PR.
With a smooth review process, a pull request usually gets reviewed
quicker.
If you don't know how to write and run your tests, please read the
guide:
https://doc.crates.io/contrib/tests
### Additional information
Other information you want to mention in this PR, such as prior arts,
future extensions, an unresolved problem, or a TODO list.
-->
This takes the most surgical, direct route to addressing the problem.
Alternatively, we could look into why `cargo rustc` and `cargo check`
are different.
Fixes#15493
### What does this PR try to resolve?
This PR adds Cargo integration for the new unstable `-Zembed-metadata`
rustc flag, which was implemented in
https://github.com/rust-lang/rust/pull/137535 ([tracking
issue](https://github.com/rust-lang/rust/issues/139165)). The new
behavior has to be enabled explicitly using a new unstable CLI flag
`-Zno-embed-metadata`.
The `-Zembed-metadata=no` rustc flag can reduce disk usage of compiled
artifacts, and also the size of Rust dynamic library artifacts shipped
to users. However, it is not enough to just pass this flag through
`RUSTFLAGS`; it needs to be integrated within Cargo, because it
interacts with how the `--emit` flag is passed to rustc, and also how
`--extern` args are passed to the final linked artifact build by Cargo.
Furthermore, using the flag for all crates in a crate graph compiled by
Cargo would be suboptimal (this will all be described below).
When you pass `-Zembed-metadata=no` to rustc, it will not store Rust
metadata into the compiled artifact. This is important when compiling
libs/rlibs/dylibs, since it reduces their size on disk. However, this
also means that everytime we use this flag, we have to make sure that we
also:
- Include `metadata` in the `--emit` flag to generate a `.rmeta` file,
otherwise no metadata would be generated whatsoever, which would mean
that the artifact wouldn't be usable as a dependency.
- Pass also `--extern <dep>=<path>.rmeta` when compiling the final
linkable artifact. Before, Cargo would only pass `--extern
<dep>=<path>.[rlib|so|dll]`. Since with `-Zembed-metadata=no`, the
metadata is only in the `.rmeta` file and not in the rlib/dylib, this is
needed to help rustc find out where the metadata lies.
- Note: this essentially doubles the cmdline length when compiling the
final linked artifact. Not sure if that is a concern.
The two points above is what this PR implements, and why this rustc flag
needs Cargo integration.
The `-Zembed-metadata` flag is only passed to libs, rlibs and dylibs. It
does not seem to make sense for other crate types. The one situation
where it might make sense are proc macros, but according to @bjorn3 (who
initially came up with the idea for `-Zembed-metadata`, it isn't really
worth it).
Here is a table that summarizes the changes in passed flags and
generated files on disk for rlibs and dylibs:
| **Crate type** | **Flags** | **Generated files** | **Disk usage** |
|--|--|--|--|
| Rlib/Lib (before) | `--emit=dep-info,metadata,link` | `.rlib` (with
metadata), `.rmeta` (for pipelining) | - |
| Rlib/Lib (after) | `--emit=dep-info,metadata,link -Zembed-metadata=no`
| `.rlib` (without metadata), `.rmeta` (for metadata/pipelining) |
Reduced (metadata no longer duplicated) |
| Dylib (before) | `--emit=dep-info,link` | `[.so\|.dll]` (with
metadata) | - |
| Dylib (after) | `--emit=dep-info,metadata,link -Zembed-metadata=no` |
`[.so\|.dll]` (without metadata), `.rmeta` | Unchanged, but split
between two files |
Behavior for other target kinds/crate types should be unchanged.
From the table above, we can see two benefits of using
`-Zembed-metadata=no`:
- For rlibs/dylibs, we no longer store their metadata twice in the
target directory, thus reducing target directory size.
- For dylibs, we store esssentially the same amount of data on disk, but
the benefit is that the metadata is now in a separate .rmeta file. This
means that you can ship the dylib (`.so`/`.dll`) to users without also
shipping the metadata. This would slightly reduce e.g. the
[size](https://github.com/rust-lang/rust/pull/120855#issuecomment-1937018169)
of the shipped rustc toolchains (note that the size reduction here is
after the toolchain has been already heavily compressed).
Note that if this behavior ever becomes the default, it should be
possible to simplify the code quite a bit, and essentially merge the
`requires_upstream_objects` and `benefits_from_split_metadata`
functions.
I did a very simple initial benchmark to evaluate the space savings on
cargo itself and
[hyperqueue](https://github.com/It4innovations/hyperqueue) (a mid-size
crate from my work) using `cargo build` and `cargo build --release` with
and without `-Zembed-metadata=no`:
For debug/incremental builds, the effect is smaller, as the artifact
disk usage is dwarfed by incremental artifacts and debuginfo. But for
(non-incremental) release builds, the disk savings (and also performed
I/O operations) are significantly reduced.
### How should we test and review this PR?
I wrote two basic tests. The second one tests a situation where a crate
depends on a dylib dependency, which is quite rare, but the behavior of
this has actually changed in this PR (see comparison table above).
Testing this on various real-world projects (or even trying to enable it
by default across the whole Cargo suite?) might be beneficial.
## Unresolved questions
### Is this a breaking change?
With this new behavior, dylibs and rlibs will no longer contain
metadata. If they are compiled with Cargo, that shouldn't matter, but
other build systems might have to adapt.
### Should this become the default?
I think that in terms of disk size usage and performed I/O operations,
it is a pure win. It should either generate less disk data (for rlibs)
or the ~same amount of data for dylibs (the data will be a bit larger,
because the dylib will still contain a metadata stub header, but that's
like 50 bytes and doesn't scale with the size of the dylib, so it's
negligible).
So I think that eventually, we should just do this by default in Cargo,
unless some concerns are found. I suppose that before stabilizing we
should also benchmark the effect on compilation performance.
### What does this PR try to resolve?
- Added an error message when version in `CRATE[@<VER>]` or `--version
<VER>` starts with 'v' for `install`, `add`, `yank` and `update
--precise <VER>`
- Check if version is valid in `cargo yank`
Fixes#12331
### How should we test and review this PR?
Added tests for each subcommand
