This works by introspecting rustc's error output, using the JSON format
to determine whether it's a warning or error, then skipping it
altogether if it's a summary of the diagnostics printed.
Before:
```
src/main.rs:1:10: warning: trait objects without an explicit `dyn` are deprecated
src/main.rs:1:1: error[E0601]: `main` function not found in crate `wrong`
src/main.rs:1:9: error[E0038]: the trait `Clone` cannot be made into an object
error: aborting due to 2 previous errors; 1 warning emitted
error: could not compile `wrong`
```
After:
```
$ cargo check --message-format short
src/main.rs:1:10: warning: trait objects without an explicit `dyn` are deprecated
src/main.rs:1:1: error[E0601]: `main` function not found in crate `wrong`
src/main.rs:1:9: error[E0038]: the trait `Clone` cannot be made into an object
error: could not compile `wrong` due to 2 previous errors; 1 warning emitted
```
Stabilize cache-messages
This stabilizes the -Zcache-messages feature, making it always enabled.
## What is stabilized?
This feature is intended to redisplay previous warnings on a "fresh" build instead of displaying no output.
Users have occasionally indicated frustration when they know there are warnings, but no output is displayed when the build is fresh. This also improves the interaction between `cargo check` and `cargo clippy-preview`. This also simplifies the code, and opens more possibilities for `rustc` to send side-channel messages to Cargo.
Cargo will now use JSON output from `rustc` and `rustdoc` 100% of the time (`rustdoc --test` does not use JSON). Previously Cargo would only use JSON for pipelined crates.
Cargo will save the JSON output into a file named `output` in the `.fingerprint` directory. This file is only created when the compiler outputs a diagnostic message.
If a crate is being recompiled, and Cargo considers it to be "fresh", it will replay the output file to the console.
## Notable changes in this PR
- Fixed a bug where replays were erroneously including pipeline rmeta artifact json messages.
- clippy-preview is now included in the metadata hash, to force its artifacts to be separate from `cargo check`.
- clippy-preview is no longer force-enabled, under the assumption that caching and fingerprinting is accurate, and the cached messages will be replayed.
- clippy-preview's arguments are included in the fingerprint now that it is not force-enabled.
- Rustdoc colors and short messages were fixed when pipelining was stabilized, so updated tests.
Closes#6986Closes#6848Closes#6664Closes#2458
## Concerns
The only notable issue with this is that switching between short and long human messages only replays the format from the first invocation. That is, if you do `cargo build` and it generates warnings, then running again with `--message-format=short` will still show the full length human messages. I'm personally fine with that behavior, even though it is not ideal. I think this feature overall improves the situation (where before *no* output was displayed). Being able to re-render between short/long is a very difficult problem, and unlikely to be fixable in the foreseeable future.
There was some concern expressed about being able to disable this. I think that would only be necessary if a severe bug is discovered. I do not feel that this change is risky enough to warrant a configurable option. If it does cause a problem, it can be quickly reverted with a one-line change to set `OutputOptions::cache_cell` to `None`. Since pipelining has been using JSON output for a while now without complaints, I feel pretty confident in it.
Implement the Cargo half of pipelined compilation (take 2)
This commit starts to lay the groundwork for #6660 where Cargo will
invoke rustc in a "pipelined" fashion. The goal here is to execute one
command to produce both an `*.rmeta` file as well as an `*.rlib` file
for candidate compilations. In that case if another rlib depends on that
compilation, then it can start as soon as the `*.rmeta` is ready and not
have to wait for the `*.rlib` compilation.
Initially attempted in #6864 with a pretty invasive refactoring this
iteration is much more lightweight and fits much more cleanly into
Cargo's backend. The approach taken here is to update the
`DependencyQueue` structure to carry a piece of data on each dependency
edge. This edge information represents the artifact that one node
requires from another, and then we a node has no outgoing edges it's
ready to build.
A dependency on a metadata file is modeled as just that, a dependency on
just the metadata and not the full build itself. Most of cargo's backend
doesn't really need to know about this edge information so it's
basically just calculated as we insert nodes into the `DependencyQueue`.
Once that's all in place it's just a few pieces here and there to
identify compilations that *can* be pipelined and then they're wired up
to depend on the rmeta file instead of the rlib file.
Closes#6660
This commit fixes an issue when pipelining mode is used in handling
recompilations. Previously a sequence of compilations could look like:
* Crate A starts to build
* Crate A produces metadata
* Crate B, which depends on A, starts
* Crate B finishes
* Crate A finishes
In this case the mtime for B is before that of A, which fooled Cargo
into thinking that B needed to be recompiled. In this case, however, B
doesn't actually need to be recompiled because it only depends on the
metadata of A, not the final artifacts.
This unfortunately resulted in some duplication in a few places, but not
really much moreso than already exists between fingerprinting and compilation.
This commit starts to lay the groundwork for #6660 where Cargo will
invoke rustc in a "pipelined" fashion. The goal here is to execute one
command to produce both an `*.rmeta` file as well as an `*.rlib` file
for candidate compilations. In that case if another rlib depends on that
compilation, then it can start as soon as the `*.rmeta` is ready and not
have to wait for the `*.rlib` compilation.
Initially attempted in #6864 with a pretty invasive refactoring this
iteration is much more lightweight and fits much more cleanly into
Cargo's backend. The approach taken here is to update the
`DependencyQueue` structure to carry a piece of data on each dependency
edge. This edge information represents the artifact that one node
requires from another, and then we a node has no outgoing edges it's
ready to build.
A dependency on a metadata file is modeled as just that, a dependency on
just the metadata and not the full build itself. Most of cargo's backend
doesn't really need to know about this edge information so it's
basically just calculated as we insert nodes into the `DependencyQueue`.
Once that's all in place it's just a few pieces here and there to
identify compilations that *can* be pipelined and then they're wired up
to depend on the rmeta file instead of the rlib file.
.. with mutliple calls of:
fastmod --accept-all '\.cargo\("([^"]+)"\)\.arg\("([^"]+)"\)' '.cargo("${1} ${2}")' tests/testsuite/
until no changes are left.
* Collapse the nested cargotest::support module into the cargotest
module (merge the mod.rs's)
* Rename the cargotest module to support
* Nest the top-level hamcrest module into support
Generally that means either switching "foo" and "bar" around (reversing
the arrow), or it means push "foo" to "bar" (and sometimes "bar" to
"baz", etc..) to free up "foo".
For trivia that leaves 80/1222 outliers, therefore 93.4% of test
project use the default. :)
By rewriting the tests, with rerast (https://github.com/google/rerast),
to use the newly introduced "at" method.
First I added the following temporary function to cargotest::support:
pub fn project_foo() -> ProjectBuilder {
project("foo")
}
Then I defined the following rewrite.rs:
use cargotest::support::{ project, project_foo };
fn rule1(a: &'static str) {
replace!(project("foo") => project_foo());
replace!(project(a) => project_foo().at(a));
}
Then I ran rerast:
cargo +nightly rerast --rules_file=rewrite.rs --force --targets tests --file tests/testsuite/main.rs
Finally I searched and replaced the references to project_foo with
argument-less project (a little awkardly on macOS with a git clean).
find tests -type f -exec sed -i -e 's/project_foo/project/g' {} +
git clean -d tests
