Ever since the inception of Cargo and the advent of incremental
compilation at the crate level via Cargo, Cargo has tracked whether it
needs to recompile something at a unit level in its "dependency queue"
which manages when items are ready for execution. Over time we've fixed
lots and lots of bugs related to incremental compilation, and perhaps
one of the most impactful realizations was that the model Cargo started
with fundamentally doesn't handle interrupting Cargo halfway through and
resuming the build later.
The previous model relied upon implicitly propagating "dirtiness" based
on whether the one of the dependencies of a build was rebuilt or not.
This information is not available, however, if Cargo is interrupted and
resumed (or performs a subset of steps and then later performs more).
We've fixed this in a number of places historically but the purpose of
this commit is to put a nail in this coffin once and for all.
Implicit propagation of whether a unit is fresh or dirty is no longer
present at all. Instead Cargo should always know, irrespective of it's
in-memory state, whether a unit needs to be recompiled or not. This
commit actually turns up a few bugs in the test suite, so later commits
will be targeted at fixing this.
Note that this required a good deal of work on the `fingerprint` module
to fix some longstanding bugs (like #6780) and some serious hoops had to
be jumped through for others (like #6779). While these were fallout from
this change they weren't necessarily the primary motivation, but rather
to help make `fingerprints` a bit more straightforward in what's an
already confusing system!
Closes#6780
This picks up on the work done in PR #5683.
The extra output is only displayed with `-vv`.
The Windows output has the form `set FOO=foo && BAR=bar rustc ...` instead of
the form that suggested in #5683 to make escaping easier and since it's
simpler.
Fix renaming directory project using build scripts with cross-compiling.
The rename-protection introduced in #4818 checks for paths with a prefix of `/…/target/debug`, but this does not work for paths used during cross-compiling.
This change checks for paths with the full `OUT_DIR` prefix, and the `build/PKG/root-output` file now includes that full `OUT_DIR` instead of `/…/target/debug`.
This also includes a few other changes:
- Support fixing KIND=PATH style paths.
- Support fixing paths in metadata (like `cargo:root=…` or `cargo:include=…` which are common).
- Adds a "version" value to the metadata hash to ensure that cargo doesn't get confused with the new `root-output` file format.
Fixes#6177
cargo's extra verbose mode is useful for getting detailed information out of
builds in CI where it can be difficult to examine the build environment
after-the-fact. However, when multiple build scripts are running as part of a
build it's not always clear what output is from which build script. This patch
makes cargo prefix each line of build script output with the crate name and
version this case.
This change ensures cargo will output file paths in the expected format
(C:\foo\... on Windows, /foo/... elsewhere). Previously it would output
file:// URLs instead.
To support this change, additional changes were made to the test suite
string processing such that [ROOT] is now replaced with the appropriate
file path root for the platform.
The CWD template was also updated to use [CWD] like other replacement
templates and to do the replacement on the expected value rather than
the actual value to avoid replacing things we don't expect with CWD.
.. with mutliple calls of:
fastmod --accept-all '\.cargo\("([^"]+)"\)\.arg\("([^"]+)"\)' '.cargo("${1} ${2}")' tests/testsuite/
until no changes are left.
* Collapse the nested cargotest::support module into the cargotest
module (merge the mod.rs's)
* Rename the cargotest module to support
* Nest the top-level hamcrest module into support
Generally that means either switching "foo" and "bar" around (reversing
the arrow), or it means push "foo" to "bar" (and sometimes "bar" to
"baz", etc..) to free up "foo".
For trivia that leaves 80/1222 outliers, therefore 93.4% of test
project use the default. :)
By rewriting the tests, with rerast (https://github.com/google/rerast),
to use the newly introduced "at" method.
First I added the following temporary function to cargotest::support:
pub fn project_foo() -> ProjectBuilder {
project("foo")
}
Then I defined the following rewrite.rs:
use cargotest::support::{ project, project_foo };
fn rule1(a: &'static str) {
replace!(project("foo") => project_foo());
replace!(project(a) => project_foo().at(a));
}
Then I ran rerast:
cargo +nightly rerast --rules_file=rewrite.rs --force --targets tests --file tests/testsuite/main.rs
Finally I searched and replaced the references to project_foo with
argument-less project (a little awkardly on macOS with a git clean).
find tests -type f -exec sed -i -e 's/project_foo/project/g' {} +
git clean -d tests
Some logic which was tweaked around the dependencies of build script targets was
tweaked slightly in a way that causes cargo to stack overflow by accientally
adding a dependency loop. This commit implements one of the strategies discussed
in #5711 to fix this situation.
The problem here is that when calculating the deps of a build script we need the
build scripts of *other* packages, but the exact profile is somewhat difficult
to guess at the moment we're generating our build script unit. To solve this the
dependencies towards other build scripts' executions is added in a different
pass after all other units have been assembled. At this point we should know for
sure that all build script executions are in the dependency graph, and we just
need to add a few more edges.
Closes#5708
Fixes#5650. cc #5435
As part of my recent work on profiles, I introduced some situations where a
library can be compiled multiple times with different settings. Doctests were
greedily grabbing all dependencies for a package, regardless of which target
is was for. This can cause doctests to fail if it links multiple copies of
the same library.
One way to trigger this is `cargo test --release` if you have dependencies, a
build script, and `panic="abort"`. There are other (more obscure) ways to
trigger it with profile overrides.
This commit cops out trying to fix `rename_with_link_search_path` by simply
adding a loop on Windows to retry the operation that looks to need retrying.
We warn if a feature was specified corresponding to a dependency which
is not optional. However, a dependency can be both optional and
required, and we shouldn't warn in that case.
Fix optional deps in multiple sections
This commit fixes an issue where an optional dependency was listed multiple
times in a manifest (multiple sections). This regression was introduced by #5415
and happened because in the resolver we didn't record a `Dependency` as it was
accidentally deduplicated too soon.
The fix here was to ensure that all `Dependency` annotations make their way into
`Resolve` now that we rely on the listed `Dependency` values for correctness.
Closes#5475
This commit fixes an issue where an optional dependency was listed multiple
times in a manifest (multiple sections). This regression was introduced by #5415
and happened because in the resolver we didn't record a `Dependency` as it was
accidentally deduplicated too soon.
The fix here was to ensure that all `Dependency` annotations make their way into
`Resolve` now that we rely on the listed `Dependency` values for correctness.
Closes#5475
This reverts commit db3328f8f7b96701faa62756a0d55ed71899d894.
We get spurious errors for thouse tests even with debug info disabled,
so let's reenable it back.
I *think* the issue is that `link.exe` is generating debuginfo in the background
which keeps the file open and prevents us from deleting it, but hopefully by
disabling debug information we'll either be able to confirm or deny this
hypothesis.
Added tests for the environment variable RUSTC_LINKER that is
passed to the build script.
Also slightly improved readability of the code responsible for
passing the env var.
