The manifest has a few different ways of specifying whether a crate is a
procedural macro, and there's a `TomlTarget::proc_macro()` method to
unify these various lines. Unfortunately though we had a bug where one
location forgot to call the method and read the raw field! This led to
surprising behavior where the different ways to specify a proc macro
would have subtly different changes in behavior. The fix here in this PR
is to ensure that we access the property always via the method.
Closes#8315
Warn if using hash in git URL, Fixes#8241
This fixes an issue where if the user wants to set the git rev but doesn't know how and as results tries to set the ref in the url hash as also shown when downloading the dependency.
Now cargo returns a warning notifying the user about the correct way to set the ref.
Fixes#8241
reset lockfile information between resolutions
#8249 pointed out that some kind of lockfile data was leaking between calls to the resolver. @ehuss made a reproducing test case. This PR resets the `LockedMap` data structure when calling `register_previous_locks`.
lets see if CI likes it.
fix#8249
Add support for rustdoc root URL mappings.
This adds an experimental configuration setting to allow Cargo to pass the `--extern-html-root-url` flag to rustdoc. This flag allows rustdoc to link to other locations when a dependency is not locally documented. See the documentation in `unstable.md` for more details.
There are some known issues with this implementation:
* Rustdoc doesn't seem to know much about renamed dependencies. The links it generates are to the package name, not the renamed name. The code is written to pass in package names, but if there are multiple dependencies to the same package, it won't work properly.
* Similarly, if there are multiple versions of the same package within the dep graph, rustdoc will only link to one of them. To fix this, Cargo would need to pass metadata info into rustdoc (such as the package version).
* If a dependency is built with different features than what is on docs.rs, some links may break.
* This explodes the command-line length significantly. Before stabilizing, we may want to consider addressing that. I'm not sure if it would make sense to change rustdoc's interface, or to use response files?
* This does not pass mappings for transitive dependencies. This normally isn't an issue, but can arise for re-exports (see the `alt_registry` test for an example). I'm not sure if this is a bug in rustdoc or not (there is a large number of issues regarding reexports and rustdoc). Cargo could include these, but this would make the command-line length even longer. Not sure what to do here.
* The config value does not support environment variables. This would be very difficult to support, because Cargo doesn't retain the registry name in `SourceId`. I looked into fixing that, but it is very difficult, and hard to make it reliable.
I have tried to consider future changes in this design, to ensure it doesn't make them more difficult:
* Single-tab browsing. This would be a mode where the std docs are merged with the local crate's docs so that the std docs are shown in the same place (and included in the index). This could be expressed with something like `doc.extern-map.std = "include"` or something like that. (Or maybe just use build-std?)
* Direct-dependencies only. Often transitive dependencies aren't that interesting, and take up a lot of space in the output, and clog the search index. Some users want the ability to (locally) document their package + direct dependencies only. I think this could be implemented with some kind of command-line flag, perhaps with a config setting in the `[doc]` table. `--extern-html-root-url` flag will automatically handle second-level dependencies.
* Manual-exclusions. Sometimes there are specific dependencies that are very expensive to document locally, but you still want everything else. I think this could be implemented with a command-line flag (`--exclude winapi`?), and the rustdoc-map feature would automatically link those excluded crates' items to docs.rs. This could also be added to the `[doc]` table.
We can also consider at any time to change the defaults (such as making `crates-io = "https://docs.rs"` the default). It could also potentially auto-detect `std = "local"`, although rustdoc could do the same internally.
Closes#6279
This stops using `to_string` as a proxy for this now-provided precise API.
This reverts commit b71927224fd9306b2b5bd2b4f8c22268eadfeb6a and bupms the
dependency version in Cargo.toml.
Provide better error messages for a bad `patch`.
This attempts to provide more user-friendly error messages for some situations with a bad `patch`. This is a follow-up to #8243.
I think this more or less covers all the issues from #4678. I imagine there are other corner cases, but those will need to wait for another day. The main one I can think of is when the patch location is missing required features. Today you get a "blah was not used in the crate graph." warning, with some suggestions added in #6470, but it doesn't actually check if there is a feature mismatch.
Closes#4678
Try installing exact versions before updating
When an exact version is being installed, if we already have that
version from the index, we don't need to update the index before
installing it. Don't do this if it's not an exact version, because the
update may find us a newer version.
This is particularly useful for scripts which unconditionally run
`cargo install some-crate --version=1.2.3`. Before install-update, I
wrote a crate to do this
(https://crates.io/crates/cargo-ensure-installed) which I'm trying to
replace with just `cargo install`, but the extra latency of updating the
index for a no-op is noticeable.
Add option to strip binaries
This PR adds a Cargo option for stripping symbols from generated binaries.
This is based on the `-Z strip` flag for `rustc`, which has been [recently implemented](https://github.com/rust-lang/rust/issues/71757).
Notes for reviewers: I'm not entirely sure how we test this, I've created a crate locally and it seems to be working.
Supersedes my previous work in #8191.
Fixes#3483.
Handle LTO with an rlib/cdylib crate type
In the case that LTO is happening but we're also generating a
cdylib/rlib simultatneously that means that the final artifact will use
the rlib but the cdylib still needs to be produced. To get this to work
the cdylib's dependency tree needs to be compiled with embedded bitcode.
The cdylib itself will be linked with the linker because we can't LTO an
rlib+cdylib compilation, but the final executable will need to load
bitcode from all its deps.
This is meant to address rust-lang/rust#72268
In the case that LTO is happening but we're also generating a
cdylib/rlib simultatneously that means that the final artifact will use
the rlib but the cdylib still needs to be produced. To get this to work
the cdylib's dependency tree needs to be compiled with embedded bitcode.
The cdylib itself will be linked with the linker because we can't LTO an
rlib+cdylib compilation, but the final executable will need to load
bitcode from all its deps.
This is meant to address rust-lang/rust#72268
Rephrased error message for disallowed sections in virtual workspace
I changed the error message from `virtual manifests do not specify [target]` to `This virtual manifest specifies a [target] section, which is not allowed` because the old one confused me.
I encountered it while hacking on Rustc in CLion. I made a change to the `Cargo.toml` I had copied from StackOverflow and after the next restart code analysis and go-to-definition stopped working. After some fiddling, I tracked that down to my change in `Cargo.toml`. It took me a while because, like I said, the error message from CLion didn't make sense to me.
```
19:13 Cargo project update failed:
Execution failed (exit code 101).
/home/julian/.cargo/bin/cargo metadata --verbose --format-version 1 --all-features
stdout : error: failed to parse manifest at `/home/julian/Dokumente/Rust/Compiler/rust/Cargo.toml`
Caused by:
virtual manifests do not specify [target]
stderr :
```
I hope this change avoids future confusion :)
With `opt-level = "3"` this previously said:
must be an integer, `z`, or `s`, but found: 3 for ...
The error message doesn't make that super clear.
This should now be a bit more clear.
Fixes#8234
Turns out a case I thought didn't happen can indeed happen. Units may
depend on other units which are LTO-able because integration tests can
depend on binaries. Handle that case internally and remove a few panics.
Closes#8223
Rework rustc output file tracking.
This is some cleanup around how Cargo computes the outputs from rustc, and changes how `cargo clean -p` works. This also includes some minor observable differences detailed below.
**clean -p changes**
Previously `cargo clean -p` would build the unit graph and attempt to guess what all the filename hashes are. This has several drawbacks. It incorrectly guesses the hashes in many cases (such as different features). It also tends to cause bugs because it passes every permutation of every setting into Cargo's internals, which may not be prepared to accept unsupported combinations like "test a build-script".
This new implementation uses a strategy of querying rustc for the filename prefix/suffix, and then deletes the files using globs.
`cargo clean -p` should now be more complete in deleting a package's artifacts, for example:
- Now deletes incremental files.
- Deletes dep-info files (both rustc and cargo).
- Handles changes in profiles, features, (anything in the hash).
- Covers a few more files (for example, dSYM in some cases, etc.). Should delete almost all files for most targets.
**Internal changes**
There are a bunch of internal changes to make Cargo do a better job of tracking the outputs from rustc, and to make the code easier to understand:
- The list of output files is now solely computed in `TargetInfo`. The files which are uplifted are solely computed in `CompilationFiles::uplift_to`. Previously both responsibilities were awkwardly spread in both locations.
- Whether or not a file should have a hyphen or underscore is now determined in one place (`FileType::should_replace_hyphens`).
- Added `CrateType` to replace `LibKind`, to avoid usage of strings, and to use the same structure for all of the target kinds.
- Added `FileFlavor::Rmeta` to be more explicit as to which output files are ".rmeta". (Previously the `Linkable{rmeta}` flag was specific for pipelining, and rmeta was `false` for things like `cargo check`, which was a bit hard to deal with.)
- Removed hyphen/underscore renaming in `rustc`. This was mostly unused, because it did not consider hashes in the filename, so it only applied to binaries without hashes, which is essentially just wasm32 and msvc. This hyphen/underscore translation still happens during "uplift".
- Changed it so that `Metadata` is always computed for every unit. The logic of whether or not something should use it is moved to `should_use_metadata`. I didn't realize that multiple units were sharing the same fingerprint directory (when they don't have a hash), which caused some bugs (like bad output caching).
**Behavioral changes**
Cargo now does more complete tracking of the files generated by rustc (and the linker). This means that some files will now be uplifted that previously weren't. It also means they will show up in the artifact JSON notifications. The newly tracked files are:
- `.exp` export files for Windows MSVC dynamic libraries. I don't know if these are actually useful (nobody has asked for them AFAIK). Presumably the linker creates them for some reason. Note that lld *doesn't* generate these files, this is only link.exe.
- Proc-macros on Windows track import/export files.
- All targets (like tests, etc.) that generate separate debug files (pdb/dSYM) are tracked.
- Added `.map` files for wasm32-unknown-emscripten.
Some other subtle changes:
- A binary example with a hyphen on Windows MSVC will now show up as `examples/foo_bar.exe` and `examples/foo-bar.exe`. Previously Cargo would just rename it to contain the hyphen. This is a consequence of simplifying the code, and I was reluctant to add a special case for this very narrow situation.
- Example libs now follow the same rules for hyphen/underscore translation as normal libs (they will now use underscores).
- Fingerprint changes:
- Fingerprint files no longer have a hash in them. Their parent directory already contained the hash.
- The fingerprint filename now uses hyphens instead of converting to underscores.
- The fingerprint directory is now separated even if the unit doesn't use Metadata, to fix a caching bug.
- macOS: dSYM is uplifted for all dynamic libraries (dylib/cdylib/proc-macro) and for build-script-build (in case someone wants to debug a build script?).
**Notes**
- I suspect that the implementation of `clean -p` may change again in the future. If and when Cargo implements some kind of on-disk database that tracks artifacts (for the purpose of garbage collection), then `cargo clean -p` can be rewritten to use that mechanism if appropriate.
- The `build_script_build` incremental directory isn't deleted because Cargo doesn't know which one belongs to which package. I'm uncertain if that's reasonably fixable. The only option I've thought of is to place each package's incremental output in a separate directory.
- Should Cargo use `globset` to handle non-utf-8 filenames? I suspect that Cargo's support for non-utf-8 names is pretty poor, so I'm uncertain how important that is.
Closes#8149Closes#6937Closes#5788Closes#5375Closes#3530
When a unit does not have Metadata, the computation of fingerprints
depends on reusing the same fingerprint file to detect if the
output needs to be rebuilt. The previous change that put each unit's
fingerprint into a separate directory was wrong, and broke change
detection in this case (for example, executables on MSVC).
Instead, use a different approach to deal with compiler output caching
by using the same naming convention as the fingerprint file itself.
Don't force rustc to do codegen for LTO builds
This commit updates Cargo's implementation of LTO builds to do less work
and hopefully be faster when doing a cold build. Additionaly this should
save space on disk! The general idea is that the compiler does not need
object files if it's only going to perform LTO with some artifacts. In
this case all rustc needs to do is load bitcode from dependencies. This
means that if you're doing an LTO build generating object code for
intermediate dependencies is just wasted time!
Here Cargo is updated with more intrusive knowledge about LTO. Cargo
will now analyze the dependency graph to figure out which crates are
being compiled with LTO, and then it will figure out which dependencies
only need to have bitcode in them. Pure-bitcode artifacts are emitted
with the `-Clinker-plugin-lto` flag. Some artifacts are still used in
multiple scenarios (such as those shared between build scripts and final
artifacts), so those are not compiled with `-Clinker-plugin-lto` since
the linker is not guaranteed to know how to perform LTO.
This functionality was recently implemented in rust-lang/rust#71528
where rustc is now capable of reading bitcode from `-Clinker-plugin-lto`
rlibs. Previously rustc would only read its own format of bitcode, but
this has now been extended! This support is now on nightly, hence this
PR.
This commit updates Cargo's implementation of LTO builds to do less work
and hopefully be faster when doing a cold build. Additionaly this should
save space on disk! The general idea is that the compiler does not need
object files if it's only going to perform LTO with some artifacts. In
this case all rustc needs to do is load bitcode from dependencies. This
means that if you're doing an LTO build generating object code for
intermediate dependencies is just wasted time!
Here Cargo is updated with more intrusive knowledge about LTO. Cargo
will now analyze the dependency graph to figure out which crates are
being compiled with LTO, and then it will figure out which dependencies
only need to have bitcode in them. Pure-bitcode artifacts are emitted
with the `-Clinker-plugin-lto` flag. Some artifacts are still used in
multiple scenarios (such as those shared between build scripts and final
artifacts), so those are not compiled with `-Clinker-plugin-lto` since
the linker is not guaranteed to know how to perform LTO.
This functionality was recently implemented in rust-lang/rust#71528
where rustc is now capable of reading bitcode from `-Clinker-plugin-lto`
rlibs. Previously rustc would only read its own format of bitcode, but
this has now been extended! This support is now on nightly, hence this
PR.
Hint git-fetch-with-cli on git errors
Our team has struggled with making Cargo git dependencies work in CI, until learning about this setting.
Cargo doesn't have a dedicated system for error hints like `rustc`, so I've stuffed the hint into the error message.
