This reverts commit 71ea2e5c5fa285e8e0336d51fd03ba4a427154bf.
`Repository::discover` and `Repository::status_file` are too expenstive
to run inside a loop. And `cargo package` are doing a lot of duplicate
works for checking submodule VCS status.
The possible fix might look like
* Let `status_submodules` function returns a path entry set, so
Cargo can check whether a source file is dirty based on that.
* When listing files in `PathSource`, attach the VCS status of a
path entry assoicated with. Then subsequent operations can skip
status check entirely.
The above solutions are not trivial, and the dirtiness check is
informational only based on T-cargo conclusion, so we should be
good just reverting the change now.
Again, the caveat of this is that we can't really detect
dirty symlinks that links into a Git submodule.
If a there is a symlink into a git repository/submodule,
when checking its git status with the wrong outer repo,
we'll get an NotFound error,
as the object doesn't belong to the outer repository.
This kind of error blocked the entire `cargo package` operation.
This fix additionally discovers the nearest Git repository,
and then checks status with that,
assuming the repo is the parent of the source file of the symlink.
This is a best effort solution, so if the check fails we ignore.
Dirtiness check for symlinks is mostly informational.
And changes in submodule would fail git-status as well (see #15384).
To avoid adding complicated logic to handle that,
for now we ignore the status check failure.
Stop a few tests which aren't marked with `public_network_test` from
trying to access crates.io. While doing so, also make sure that other
`cargo_add` and `cargo_info` tests initialize the registry.
Lockfile might be gitignore'd,
So it never shows up in `git status`.
However, `cargo packag` vcs checks actually performs
`git status --untracked --ignored`.
It is a bit confusing that lockfile is ignored but still counts as dirty
from the report of `cargo package`.
There are some more nuances:
We check lockfile's VCS status if the lockfile is ouside the current
package root. That means a non-workspace Cargo package will not have
this VCS check. I don't think it is good that we have diverged behaviors
Hence this revert.
We can always re-evaluate later,
as we've reserved rooms for doing more dirty checks.
https://github.com/rust-lang/cargo/issues/14967#issuecomment-2651329783
### What does this PR try to resolve?
Fixes#15059Fixes#15159
This provides an escape hatch `--exclude-lockfile`for uncommon workflows
that don't verify (`--no-verify` is passed) the build with their
unpublished packages
In effect, this takes the heuristic removed in #14815 and replaces it
with a flag
When `--exclude-lockfile` is enabled,
`cargo package` will not verify the lock file if present,
nor will it generate a new one if absent.
Cargo.lock will not be included in the resulting tarball.
Together with `--no-verify`,
this flag decouples packaging from checking the registry index.
While this is useful for some non-normal workflows that requires
to assemble packages having unpublished dependencies.
It is recommended to use `-Zpackage-workspace` to package the entire
workspace, instead of opting out lockfile.
### How should we test and review this PR?
The first commit was stolen from
<1a104b5444>
(credit to @NoisyCoil!)
The second added two failing cases we observed in #15059.
### Additional information
Lockfile might not be under the package root, but its entries
may be outdated and get packaged into the `.crate` tarball.
We take a conservative action that if the lockfile is dirty,
then all workspace members are considered dirty.
Workspace manifest might not be under the package root, but
workspace inheritance may still affect what is packaged into
the `.crate.` tarball. We take a conservative action that if
the workspace manifest is dirty, then all workspace members
are considered dirty.
### What does this PR try to resolve?
Just found we can simplify `human_readable_bytes` a bit.
This also remove the `bytesize` dependency,
since we already have a hand-rolled function.
(It is a good crate, though)
### How should we test and review this PR?
Should have no real functional difference.
`cargo package` starts reporting IEC (KiB) instead of SI (KB).
So do some thresholds of `cargo package`.
Could also switch entirely to `bytesize` if that is preferred.
### Additional information
When `--exclude-lockfile` is enabled,
`cargo package` will not verify the lock file if present,
nor will it generate a new one if absent.
Cargo.lock will not be included in the resulting tarball.
Together with `--no-verify`,
this flag decouples packaging from checking the registry index.
While this is useful for some non-normal workflows that requires
to assemble packages having unpublished dependencies.
It is recommended to use `-Zpackage-workspace` to package the entire
workspace, instead of opting out lockfile.
After dd698ff0483bda065d1f78968e70ee86fb116a72,
`cargo package --no-verify` at least fails in three different cases:
* An unpublished package depending on itself as a dev-dependency (cyclic self-referential dev-dependencies).
* Can be resolved by removing the `version` field from the affected dev-dependency.
* `-Zpackage-workspace` doesn't help with it.
* Existing `cargo package` has `--package <pkg>` specifying certain unpublished packages.
* Can be resolved by specifying all unpublished packages in one `cargo` call.
* `-Zpackage-workspace` also requires all dependency versions available in the target registry when calling, so doesn't help.
* `cargo package --no-verify` has been used as a kind of “plumbing commands” to create tarballs without considering dependency orders. The use cases include:
* Preparing tarballs for other package managers.
* Integrating into custom develop workflows for unpublished/internal crates.
* Constructing custom/private registries.
This commit shows the former two cases.
metdata path fields may point to a dirty symlilnk and cause
duplicate report. This commit combines `dirty_metadata_paths`
and `dirty_symlinks` into one so is able to de-duplicate them.
This adds a special case for checking source files are symlinks
and have been modified when under a VCS control
This is required because those paths may link to a file outside the
current package root, but still under the git workdir, affecting the
final packaged `.crate` file.
This may have potential performance issue. If a package contains
thousands of symlinks, Cargo will fire `git status` for each of them.
`cargo package` will warn users when git `core.symlinks` is `false`
and some symlinks were checked out as plain files during packaging.
Git config [`core.symlinks`] defaults to true when unset.
In git-for-windows (and git as well),
the config should be set to false explicitly when the repo was created,
if symlink support wasn't detected [^1].
We assume the config was always set at creation time and never changed.
So, if it is true, we don't bother users with any warning.
[^1]: <f1241afcc7/setup.c (L2394-L2403)>
[`core.symlinks`]: https://git-scm.com/docs/git-config#Documentation/git-config.txt-coresymlinks
This adds a special case for `package.{readme,license-file}`
to Git VCS status check.
If they were specified with paths outside the current package root,
but still under git workdir, Cargo checks git status of those files
to determine if they were dirty.
We don't need to take care of other fields with path values because
* `PathSource` only list files under the package root.
Things like `target.path` works for `cargo build`, but won't be
included in `.crate` file from `cargo publish`.
* The only exceptions are `package.readme`/`package.license-file`.
Cargo would copy files over if they are outside package root.
This is a test showing corner cases that dirty files outside
the package being packaging actually made the `.crate` file dirty.
However, `cargo package` and `.cargo_vcs_info.json` didn't capture it.
Personally I liked that the test was only dependent on what really matters, the lack of presence of a particular filename. Now the test would fail if Cargo one day adds more (generated) files to the package.
Co-authored-by: Weihang Lo <weihanglo@users.noreply.github.com>
This is a follow up to #14846 which changed `run` to return the
`RawOutput`.
Reasons I didn't "update" some code to the new `run` return value
- We were actually using `ProcessBuilder::exec_with_output` and I didn't
want to disentangle what it would take to switch to `Execs`
- We did processing on the `Result` and I didn't want to check how that
could be updated
Originally it was only included for packages that have executables or
examples for `cargo install`, however this causes inconsistencies and
is kind of unexpected nowadays, e.g. with cdylib crates.
Including it always only slightly increases the crate size and allows
for all crates to know a set of dependency versions that were working,
which can make regression tracking easier.
Fixes https://github.com/rust-lang/cargo/issues/13447
`Registry::describe_source` will be used as default error message, but if the message
come from `Registry::describe_source` is empty, then it uses `SourceId` message
PR #5335 added `autobins`, etc for #5330. Nowhere in there is
discussion of `autolib`.
Cargo script disables support for additional build-targets by disabling
discovery.
Except we don't have a way to disable discovery of `autolib`, leading to #14476.
By adding `autolib`, we can continue in that direction.
This also allows us to bypass inferring of libs on published packages,
like all other build-targets which were handled in #13849.
As this seems fairly low controversy, this insta-stabilizes the field.
In prior versions of Cargo, users will get an "unused manifest key"
warning.
For packags where this is set by `cargo publish`, the warning will be suppressed and things will work as normal.
For `cargo vendor`, the same except there will be some churn in the
vendored source as this field will now be set.
For local development, it should be rare to set `autolib` so the lack of
error by discovering a file when this is set shouldn't be a problem.
Fixes#14476
This commit makes lockfile version 4 the default version when Cargo
tries to write to a lockfile.
The lockfile version 4 has been stabilized since 1.78.0,
and will become default in 1.83.0.
the length of transition period is pretty similar as before.
One caveat is that in other output from Cargo,
e.g., `cargo metatada`, status messages,
`SourceID` will display in the v4 URL encoded format.
This shouldn't affect the majority of Rust users,
as `SourceId` representation should be opaque to them,
unless comparing `SourceId` across different version of toolchains.
