There's no need to try every candidate for every dependency - instead,
only try alternative candidates if they *could* change the eventual
failure that caused backtracking in the first place.
This commit is an attempt to stem the tide of "cargo is stuck updating the
registry" issues by giving a better indication as to what's happening in
long-running steps. The primary addition here is a `Progress` helper module
which prints and manages a progress bar for long-running operations like git
fetches, git checkouts, HTTP downloads, etc.
The second addition here is to print out when we've been stuck in resolve for
some time. We never really have a progress indicator for crate graph resolution
nor do we know when we're done updating sources. Instead we make a naive
assumption that when you've spent 0.5s in the resolution loop itself (not
updating deps) you're probably done updating dependencies and on to acutal
resolution. This will print out `Resolving crate graph...` and help inform that
Cargo is indeed not stuck looking at the registry, but rather it's churning away
in resolution.
Fixes#4326
`cargo package` (and so `cargo publish`) parses a crate’s `Cargo.toml`,
makes some modifications, and re-serializes it.
Because the `TomlManifest` struct uses `HashMap`
with its default `RandomState` hasher,
the maps’ iteration order changed on every run.
As a result, when using `cargo vendor`,
updating a dependency would generate a diff larger than necessary,
with non-significant order-changes obscuring significant changes.
This replaces some uses of `HashMap` with `BTreeMap`,
whose iteration order is deterministic (based on `Ord`).
Currently the vendoring support in Cargo primarily only allows replacing
registry sources, e.g. crates.io. Other networked sources of code, such as git
repositories, cannot currently be replaced. The purpose of this commit is to
support vendoring of git dependencies to eventually have support implemented in
the `cargo-vendor` subcommand.
Support for vendoring git repositories required a few subtle changes:
* First and foremost, configuration for source replacement of a git repository
was added. This looks similar to the `Cargo.toml` configuration of a git
source.
* The restriction around checksum providing sources was relaxed. If a
replacement source provides checksums but the replaced source doesn't then
that's now considered ok unlike it being an error before.
* Lock files can be generated for crates.io crates against vendored sources, but
lock files cannot be generated against git sources. A lock file must
previously exist to make use of a vendored git source.
* The `package` field of `.cargo-checksum.json` is now optional, and it is
intended to be omitted for git sources that are vendored.
This commit is a relatively serious optimization pass of the resolution phase in
Cargo, targeted at removing as many allocations as possible from this phase.
Executed as an iterative loop this phase of Cargo can often be costly for large
graphs but it's run on every single build!
The main optimization here is to avoid cloning the context and/or pushing a
backtracking frame if there are no candidates left in the current list of
candidates. That optimizes a fast-path for crates with lock files (almost all of
them) and gets us to the point where cloning the context basically disappears
from all profiling.
This commit is an implementation of [RFC 1525] which specifies the addition of
**workspaces** to Cargo.
[RFC 1525]: https://github.com/rust-lang/rfcs/blob/master/text/1525-cargo-workspace.md
A workspace is a group of crates which are all compiled into the same output
directory and share the same `Cargo.lock` file. This means that dependencies are
cached between builds as well as dependencies all being shared at the same
versions. An update to any one dependency transitively affects all other members
of the workspace.
Typical repository layouts with a crate at the root and a number of path
dependencies simply need to add the following to the root `Cargo.toml`:
```toml
[workspace]
```
Otherwise more advanced configuration may be necessary through the
`package.workspace` or `workspace.members` keys. More information can be found
as part of [RFC 1525].
This commit is an implementation of top-level overrides to be encoded into the
manifest itself directly. This style of override is distinct from the existing
`paths` support in `.cargo/config` in two important ways:
* Top level overrides are intended to be checked in and shared amongst all
developers of a project.
* Top level overrides are reflected in `Cargo.lock`.
The second point is crucially important here as it will ensure that an override
on one machine behaves the same as an override on another machine. This solves
many long-standing problems with `paths`-based overrides which suffer from some
level of nondeterminism as they're not encoded.
From a syntactical point of view, an override looks like:
```toml
[replace]
"libc:0.2.0" = { git = 'https://github.com/my-username/libc', branch = '0.2-fork' }
```
This declaration indicates that whenever resolution would otherwise encounter
the `libc` package version 0.2.0 from crates.io, it should instead replace it
with the custom git dependency on a specific branch.
The key "libc:0.2.0" here is actually a package id specification which will
allow selecting various components of a graph. For example the same named
package coming from two distinct locations can be selected against, as well as
multiple versions of one crate in a dependency graph. The replacement dependency
has the same syntax as the `[dependencies]` section of Cargo.toml.
One of the major uses of this syntax will be, for example, using a temporary
fork of a crate while the changes are pushed upstream to the original repo. This
will avoid the need to change the intermediate projects immediately, and over
time once fixes have landed upstream the `[replace]` section in a `Cargo.toml`
can be removed.
There are also two crucial restrictions on overrides.
* A crate with the name `foo` can only get overridden with packages also of
the name `foo`.
* A crate can only get overridden with a crate of the exact same version.
A consequence of these restrictions is that crates.io cannot be used to replace
anything from crates.io. There's only one version of something on crates.io, so
there's nothing else to replace it with (name/version are a unique key).
Closes#942
Previously it was a mix of vec!() and vec![], as both work thanks to the
implementation of macros in Rust (even vec!{} would work), this makes
them all use the generally-accepted vec![] standard, which imitates the
syntax used for slices and is considered to make things clearer.
Resolving a dependency graph may involve quite a bit of backtracking to select
different versions of crates, but the previous implementation of resolution
would not backtrack enough.
Once a dependency is completely resolved it's possible that any number of
candidates for its transitive dependencies were left out of the resolution
process (e.g. we didn't hit them yet). These candidates were not previously used
for backtracking (because resolution overall of the dependency finished), but
this commit alters the implementation to instead consider these as candidates
for backtracking.
Architecturally this changes the code to CPS to pass around a `finished`
continuation to allow tweaking the behavior whenever a dependency successfully
resolves. The key change is then that whenever a candidate for a dependency is
activated, we ensure the recursive step for the rest of the graph happens as a
sub-call. This means that if *anything* in the recursive call fails (including
some previous up-the-stack resolution) we'll retry the next candidate.
Closes#1800
The new message will print out a selection of versions that were found if any
are to be had, as well as a recommendation to run `cargo update` if it's a path
dependency.
Closes#1145
This commit unifies the notion of a "git revision" between a SourceId and the
GitSource. This pushes the request of a branch, tag, or revision all the way
down into a GitSource so special care can be taken for each case.
This primarily was discovered by #1069 where a git tag's id is different from
the commit that it points at, and we need to push the knowledge of whether it's
a tag or not all the way down to the point where we resolve what revision we
want (and perform appropriate operations to find the commit we want).
Closes#1069
I'm a bit shaky on the profile.rs changes (`thread_local!` and `RefCell`
relplacing `local_data_key!`), do make sure I haven't royally screwed
something up there.
Note that I haven't sucessfully run the various test_cargo_cross_compile
tests as I don't have an i686-unknown-linux-gnu rustc sitting around.
If we're activating an already-active version of a dependency, then there's no
need to actually recurse into it. Instead, we can skip it if we're not enabling
any extra features in it to avoid extraneous recursion.
This commit extends the support in cargo's resolver to start resolving packages
with multiple versions as well as package requirements. This is a crucial step
forward when impelmenting the cargo registry as multiple versions will be
uploaded to the registry quite quickly!
This implements a fairly naive solution which should at least help cargo get out
the gates initially. This impelments a depth-first-search of the pacakage
dependency graph with a few sorting heuristics along the way to help out
resolution as it goes along.
Resolution errors will likely improve over time, but this commit does make an
effort to try to get some good error messages right off the bat.
