This is meant to fixes#13191
As git sources and registry sources are considered immutable.
I don't think there is any reason excluding those files.
There might be a little chance local Git repositories might have those,
though that is a rare use case.
Alternatively,
we could reject all `.rej`/`.orig` files but `Cargo.toml.orig`.
`PathSource::list_files` has some heurstic rules for listing files.
Those rules are mainly designed for `cargo package`.
Previously, cargo-vendor relies on those rules to understand what
files to vendor. However, it shouldn't use those rules because:
* Package extracted from a `.crate` tarball isn't Git-controlled,
some rules may apply differently.
* The extracted package already went through `PathSource::list_files`
during packaging. It should be clean enough.
* Should keep crate sources from registry sources in a pristine state,
which is exactly what vendoring is meant for.
Instead, we switch to direct extraction into the vendor directory
to ensure source code is the same as in the `.crate` tarball.
There are some caveats:
* The overwrite protection in `unpack_package` assumes the unpack
directory is always `<pkg>-<version`>.
We don't want to remove this,
but for cargo-vendor supports vendoring without version suffix.
For that case, we need a temporary staging area,
and move the unpacked source then.
* The heurstic in `PathSource::list_files` did something "good" in
general cases, like excluding hidden directories. That means
common directorys like `.github` or `.config` won't be vendored.
After this, those get included. This is another round of churns.
We might want to get other `cargo-vendor` changes along with this
in one single release.
cargo-vendor has a workaround that to mitigate rust-lang/cargo#5956,
it removes all cached sources in order to trigger a re-unpack.
It was meant for dealing with registry sources only, but accidentally
applied to directory source kind.
While directory source kind was invented for vendoring,
and vendoring from one vendored directory to the other seems unusual,
Cargo IMO should not delete any real sources.
It does not mean that registry sources are okay to delete,
In long term, we should explore a way that unpacks `.crate` files
directly, without any removal. See
https://github.com/rust-lang/cargo/pull/12509#issuecomment-1732415990
This is a follow up to #14846 which changed `run` to return the
`RawOutput`.
Reasons I didn't "update" some code to the new `run` return value
- We were actually using `ProcessBuilder::exec_with_output` and I didn't
want to disentangle what it would take to switch to `Execs`
- We did processing on the `Result` and I didn't want to check how that
could be updated
PR #5335 added `autobins`, etc for #5330. Nowhere in there is
discussion of `autolib`.
Cargo script disables support for additional build-targets by disabling
discovery.
Except we don't have a way to disable discovery of `autolib`, leading to #14476.
By adding `autolib`, we can continue in that direction.
This also allows us to bypass inferring of libs on published packages,
like all other build-targets which were handled in #13849.
As this seems fairly low controversy, this insta-stabilizes the field.
In prior versions of Cargo, users will get an "unused manifest key"
warning.
For packags where this is set by `cargo publish`, the warning will be suppressed and things will work as normal.
For `cargo vendor`, the same except there will be some churn in the
vendored source as this field will now be set.
For local development, it should be rare to set `autolib` so the lack of
error by discovering a file when this is set shouldn't be a problem.
Fixes#14476
This is for `cargo generate-lockfile` and when syncing the lockfile with
the manifest.
We still show it for `cargo update` because of `cargo update
--workspace`.
We hacked around this previously by filtering out the `num_pkgs==1` case
for single packages but this didn't help with workspaces.
This is a **very** hacky solution, duplicating the minimum of what
`prepare_for_publish` does to fix this one issue and in the least
intrusive way to the vendor code.
The intention is to keep this low risk for backporting to beta and
stable.
We need to revisit this, refactoring the `cargo package` code so that we
can call into that for each vendored dependency.
Fixes#14348
Since we already cover this for `cargo package` and we turn all
implicit targets into explicit targets (making implicit tests cover
explicit cases), this becomes redundant.
While this is noisy and hides other deprecations, I figured deprecations would
make it easier for people to discover what tasks remain and allow us to
divide and conquer this work rather than doing a heroic PR.
In theory, this will be short lived and we'll go back to seeing
deprecations in our tests.
This is to help with #9930
Example changes:
```diff
-[LOCKING] 4 packages
+[LOCKING] 4 packages to latest version
-[LOCKING] 2 packages
+[LOCKING] 2 packages to latest Rust 1.60.0 compatible versions
-[LOCKING] 2 packages
+[LOCKING] 2 packages to earliest versions
```
Benefits
- The package count is of "added" packages and this makes that more
logically clear
- This gives users transparency into what is happening, especially with
- what rust-version is use
- the transition to this feature in the new edition
- whether the planned config was applied or not (as I don't want it to
require an MSRV bump)
- Will make it easier in tests to show what changed
- Provides more motiviation to show this message in `cargo update` and
`cargo install` (that will be explored in a follow up PR)
This does come at the cost of more verbose output but hopefully not too
verbose. This is why I left off other factors, like avoid-dev-deps.
Without this, an attacker can leverage globally writable files buried
in the `.crate` file. After a user downloaded and unpacked the file,
the attacker can then write malicous code to the downloaded sources.
This fixes an issue the gecko developers noticed when vendoring
on windows. [0] If a user has `core.autocrlf=true` set
(a reasonable default on windows), vendoring from a git source
would cause all the newlines to be rewritten to include carriage
returns, creating churn and platform-specific results.
To fix this, we simply set the global cargo checkout's "local"
core.autocrlf value before performing a `reset`. This masks out
the system configuration without interfering with the user's
own system/project settings.
[0]: https://bugzilla.mozilla.org/show_bug.cgi?id=1647582
This commit is targeted at further improving the error messages
generated from git errors. For authentication errors the actual URL
fetched is now printed out as well if it's different from the original
URL. This should help handle `insteadOf` logic where SSH urls are used
instead of HTTPS urls and users can know to track that down.
Otherwise the logic about recommending `net.git-fetch-with-cli` was
tweaked a bit and moved to the same location as the rest of our error
reporting.
Note that a change piggy-backed here as well is that `Caused by:` errors
are now automatically all tabbed over a bit instead of only having the
first line tabbed over. This required a good number of tests to be
updated, but it's just an updated in renderings.
Implement --explicit-version from standalone cargo-vendor. This helps with
vendoring performance as it avoids redundantly deleting and re-copying
already vendored packages.
For example, when re-vendoring cargo's dependencies it makes a big
improvement on wallclock time. For initial vendoring it makes no
difference, but re-vendoring (ie, when most or all dependencies haven't
changed) without explicit versions is actually slightly slower (5.8s ->
6s), but with explicit versions it goes from 5.8s -> 1.6s.
Timings:
Without explicit versions, initial vendor
real 0m5.810s
user 0m0.924s
sys 0m2.491s
Re-vendor:
real 0m6.083s
user 0m0.937s
sys 0m2.654s
With explicit versions, initial vendor:
real 0m5.810s
user 0m0.937s
sys 0m2.461s
Re-vendor:
real 0m1.567s
user 0m0.578s
sys 0m0.967s
The summaries of syscalls executed shows why:
Revendoring without explicit versions:
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
25.17 1.104699 18 59432 1065 openat
19.86 0.871574 21 41156 13825 unlink
13.64 0.598739 2 210510 lstat
9.02 0.395948 29 13208 copy_file_range
8.00 0.351242 11 30245 read
6.36 0.279005 3 72487 4476 statx
5.35 0.235027 6 37219 write
4.02 0.176267 3 58368 close
```
with explicit versions:
```
29.38 0.419068 15 27798 13825 unlink
25.52 0.364021 1 209586 lstat
20.67 0.294788 16 17967 1032 openat
10.42 0.148586 4 35646 write
3.53 0.050350 3 13825 chmod
3.14 0.044786 2 16701 1622 statx
2.19 0.031171 1 16936 close
1.86 0.026538 24 1078 rmdir
```
Specifically, there are a lot fewer opens, copy_file_ranges, and unlinks.
