Avoid buffering large amounts of rustc output.
If `rustc` prints out a lot of information (such as with `RUSTC_LOG`, or a huge number of diagnostics), cargo would buffer up large amounts of that in memory. For normal builds, this would happen if the terminal does not print fast enough. For "fresh" replay, *everything* was being buffered.
There are two issues:
1. There is no back-pressure on the mpsc queue. If messages come in faster than they can be processed, it grows without bounds.
2. The cache-replay code runs in the "fresh" code path which does not spawn a thread. Thus the main thread was blocked and unable to process `Message`s while the replay is happening.
The solution here is to use a bounded queue, and to always spawn a thread for the "fresh" case.
The main concern here is performance. Previously the "fresh" jobs avoided spawning a thread to improve performance. I did a fair bit of profiling to understand the impact, using projects with anywhere from 100 to 500 units. On my macOS machine, I found spawning a thread to be slightly faster (1-5%). On Linux and Windows, it was generally about 0 to 5% slower. It might be helpful for others to profile it on their own system.
I'm on the fence for the cost/benefit here. It seems generally good to reduce memory usage, but the slight performance hit is disappointing. I tried several other approaches to fix this, all with worse trade offs (I can discuss them if interested).
Fixes#6197
Add "Updating" status for git submodules.
This adds a status message when updating a git submodule. Downloading these can be very slow (often submodules are much larger than their parents). I think it is helpful to provide some more feedback as to what it is doing.
Support old html anchors in manifest chapter.
#7733 unintentionally broke some old HTML anchors in the manifest chapter. This would cause any links out in the wild to not scroll to the correct position.
Unfortunately it is too late for the 1.42 release. However, I'd like to backport this for 1.43.
We don't need the complexity of most channels since this is not a
performance sensitive part of Cargo, nor is it likely to be so any time
soon. Coupled with recent bugs (#7840) we believe in `std::sync::mpsc`,
let's just not use that and use a custom queue type locally which should
be amenable to a blocking push soon too.
Don't create hardlink for library test and integrations tests, fixing #7960
Related issue: #7960
Problem:
Tests are run under deps, but it is still copied to its parent directory. It leads to separation between the executable and its debug symbols (.dSYM directory).
Solution:
Set hardlink to None.
Partially revert change to filter debug_assertions.
This partially reverts the changes from #7943. It caused a regression with the rocket_contrib crate. I knew that was the only crate that had a `cfg(debug_assertions)` dependency, and I saw that it had been fixed, but I did not realize the fix hadn't been published (and will be in a semver incompatible release).
This retains the old behavior for `cfg(debug_assertions)` of issuing a warning. I kept the filter for `CARGO_CFG_DEBUG_ASSERTIONS` for build scripts because that was the original intent for the change, and I don't see anyone using that.
Closes#7966.
Try to better handle restricted crate names.
This attempts to improve handling of restricted crate names, particularly for `cargo new` and `cargo init`. Hopefully the code is straightforward to follow, but in summary the changes are:
**General changes**
* Add more details to the error messages about why a name is not allowed, and what is allowed.
* Change the valid package name check to be restricted to Unicode XID. This brings it in line with non_ascii_idents support in rustc. For the most part, this is pretty much the same as before. Note: this is used for the package name, and registry names. The differences are:
* Package names cannot start with numbers. Previously this was only rejected in `cargo new`. crates.io also rejects numbers. Numbers are also not valid crate names.
* Package names cannot start with dash `-`. This is a somewhat arbitrary change, but seems like it would stem problems. crates.io also rejects this.
* Package names cannot start with these characters that were previously allowed: https://gist.github.com/ehuss/804a797950001b5226e1264b6f65211f#file-not_start_but_alphanumeric-txt
* Most of these are wacky numbers or other strange things.
* Package names cannot contain these characters that were previously allowed: https://gist.github.com/ehuss/804a797950001b5226e1264b6f65211f#file-not_continue_but_alphanumeric-txt
* These are mostly odd things that for whatever reason the Unicode people decided not to include. It seems unlikely to me that someone would want to use one of these.
* Display a warning on Windows if a Cargo target is a special Windows filename. The build error tends to be hard to understand, so the hope is the warning will make it evident.
* `cargo package/publish`: Warn if a special Windows name is in the package.
**cargo new/init specific changes**
* Update keyword list to 2018 edition.
* Add warning if creating a library that has one of the conflicting names (deps/examples/build/incremental).
* Warn about conflicting std names (core/std/alloc/proc-macro).
* Windows reserved names: Rejected on windows, warned on other platforms.
* Warn about non-ASCII package names.
* Only print the `--name` suggestion for `cargo init`. I found the suggestion confusing, and I feel like it doesn't really make sense for `cargo new` (since it would only affect the directory name).
Fix bug with new feature resolver and required-features.
If required-features are used, then the code for checking those features would crash if a dependency was not activated. The solution here is to not be strict about only requesting activated packages.
For context, the reason this can panic is to check for any bugs in the resolver or places that make bad assumptions. I missed this particular case, though.
Ignore broken Cargo.toml in git sources
Commit 3d6de4177489a5d450f35e92288512be85492678 (#3998) made cargo
ignore Cargo.toml files that are invalid TOML in a git source.
This change further ignores Cargo.toml files that are valid TOML but
cannot really be loaded in a git source.
This is potentially an alternative fix for #6822.
Add more fingerprint mtime debug logging.
Adding some more debug logging on top of #7888.
There was a mistake in the original PR, where `dep.pkg_id` should have been `dep.name`.
Commit 3d6de4177489a5d450f35e92288512be85492678 (#3998) made cargo
ignore Cargo.toml files that are invalid TOML in a git source.
This change further ignores Cargo.toml files that are valid TOML but
cannot really be loaded.
This is potentially an alternative fix for #6822.
Simplified usage code of SipHasher
SipHasher::new_with_keys(0,0) is just a longer version of just _::new()
i.e. the latter is an alias for the former.
Just a bit of a nothing burger I noticed while debugging some other issue.
Fixes issue #7543
Added a helper function for `util::hex::hash_64` that uses streams
the content instead of reading through the entire content in one
go.
Edit: Should I add test cases for this?
Edit2: Per #7543
