`-Zhigher-ranked-assumptions`: Consider WF of coroutine witness when proving outlives assumptions
### TL;DR
This PR introduces an unstable flag `-Zhigher-ranked-assumptions` which tests out a new algorithm for dealing with some of the higher-ranked outlives problems that come from auto trait bounds on coroutines. See:
* rust-lang/rust#110338
While it doesn't fix all of the issues, it certainly fixed many of them, so I'd like to get this landed so people can test the flag on their own code.
### Background
Consider, for example:
```rust
use std::future::Future;
trait Client {
type Connecting<'a>: Future + Send
where
Self: 'a;
fn connect(&self) -> Self::Connecting<'_>;
}
fn call_connect<C>(c: C) -> impl Future + Send
where
C: Client + Send + Sync,
{
async move { c.connect().await }
}
```
Due to the fact that we erase the lifetimes in a coroutine, we can think of the interior type of the async block as something like: `exists<'r, 's> { C, &'r C, C::Connecting<'s> }`. The first field is the `c` we capture, the second is the auto-ref that we perform on the call to `.connect()`, and the third is the resulting future we're awaiting at the first and only await point. Note that every region is uniquified differently in the interior types.
For the async block to be `Send`, we must prove that both of the interior types are `Send`. First, we have an `exists<'r, 's>` binder, which needs to be instantiated universally since we treat the regions in this binder as *unknown*[^exist]. This gives us two types: `{ &'!r C, C::Connecting<'!s> }`. Proving `&'!r C: Send` is easy due to a [`Send`](https://doc.rust-lang.org/nightly/std/marker/trait.Send.html#impl-Send-for-%26T) impl for references.
Proving `C::Connecting<'!s>: Send` can only be done via the item bound, which then requires `C: '!s` to hold (due to the `where Self: 'a` on the associated type definition). Unfortunately, we don't know that `C: '!s` since we stripped away any relationship between the interior type and the param `C`. This leads to a bogus borrow checker error today!
### Approach
Coroutine interiors are well-formed by virtue of them being borrow-checked, as long as their callers are invoking their parent functions in a well-formed way, then substitutions should also be well-formed. Therefore, in our example above, we should be able to deduce the assumption that `C: '!s` holds from the well-formedness of the interior type `C::Connecting<'!s>`.
This PR introduces the notion of *coroutine assumptions*, which are the outlives assumptions that we can assume hold due to the well-formedness of a coroutine's interior types. These are computed alongside the coroutine types in the `CoroutineWitnessTypes` struct. When we instantiate the binder when proving an auto trait for a coroutine, we instantiate the `CoroutineWitnessTypes` and stash these newly instantiated assumptions in the region storage in the `InferCtxt`. Later on in lexical region resolution or MIR borrowck, we use these registered assumptions to discharge any placeholder outlives obligations that we would otherwise not be able to prove.
### How well does it work?
I've added a ton of tests of different reported situations that users have shared on issues like rust-lang/rust#110338, and an (anecdotally) large number of those examples end up working straight out of the box! Some limitations are described below.
### How badly does it not work?
The behavior today is quite rudimentary, since we currently discharge the placeholder assumptions pretty early in region resolution. This manifests itself as some limitations on the code that we accept.
For example, `tests/ui/async-await/higher-ranked-auto-trait-11.rs` continues to fail. In that test, we must prove that a placeholder is equal to a universal for a param-env candidate to hold when proving an auto trait, e.g. `'!1 = 'a` is required to prove `T: Trait<'!1>` in a param-env that has `T: Trait<'a>`. Unfortunately, at that point in the MIR body, we only know that the placeholder is equal to some body-local existential NLL var `'?2`, which only gets equated to the universal `'a` when being stored into the return local later on in MIR borrowck.
This could be fixed by integrating these assumptions into the type outlives machinery in a more first-class way, and delaying things to the end of MIR typeck when we know the full relationship between existential and universal NLL vars. Doing this integration today is quite difficult today.
`tests/ui/async-await/higher-ranked-auto-trait-11.rs` fails because we don't compute the full transitive outlives relations between placeholders. In that test, we have in our region assumptions that some `'!1 = '!2` and `'!2 = '!3`, but we must prove `'!1 = '!3`.
This can be fixed by computing the set of coroutine outlives assumptions in a more transitive way, or as I mentioned above, integrating these assumptions into the type outlives machinery in a more first-class way, since it's already responsible for the transitive outlives assumptions of universals.
### Moving forward
I'm still quite happy with this implementation, and I'd like to land it for testing. I may work on overhauling both the way we compute these coroutine assumptions and also how we deal with the assumptions during (lexical/nll) region checking. But for now, I'd like to give users a chance to try out this new `-Zhigher-ranked-assumptions` flag to uncover more shortcomings.
[^exist]: Instantiating this binder with infer regions would be incomplete, since we'd be asking for *some* instantiation of the interior types, not proving something for *all* instantiations of the interior types.
Unify `CoroutineWitness` sooner in typeck, and stall coroutine obligations based off of `TypingEnv`
* Stall coroutine obligations based off of `TypingMode` in the old solver.
* Eagerly assign `TyKind::CoroutineWitness` to the witness arg of coroutines during typeck, rather than deferring them to the end of typeck.
r? lcnr
This is part of https://github.com/rust-lang/rust/issues/143017.
Only inherit local hash for paths
`DefPathHash`, as the counterpart of `DefId` that is stable across compiler invocations, is comprised of 2 parts. The first one is the `StableCrateId`, stable form of `CrateNum`. The second is 64 complementary bits to identify the crate-local definition.
The current implementation always hashes the full 128 bits when (1) trying to create a new child `DefPathHash` or (2) hashing a `CrateNum` or a `LocalDefId`. But we only need half that information: `LocalDefId` means that the `StableCrateId` is always the current crate's ; `CrateNum` means that we do not care about the local part.
As stable hashing is very hot in the query system, in particular hashing definitions, this is a big deal.
We still want the local part to change when the `StableCrateId` changes, to make incr-compilation errors less painful, ie. increase the likelihood that if will magically disappear by changing some code.
This PR sprinkles some `#[inline]` attributes on small functions that appeared in profiles.
`std::vec`: Add UB check for `set_len`, `from_raw_parts_in`, and etc.
Closesrust-lang/rust#143813
I noticed that `from_parts_in` do the similar things like `from_raw_parts_in`, so I add the UB check in the last commit. If it is not appropriate, I will remove it.
And I fix a typo in the first commit.
r? `@scottmcm`
core: Add `BorrowedCursor::with_unfilled_buf`
Implementation of https://github.com/rust-lang/libs-team/issues/367.
This mainly adds `BorrowedCursor::with_unfilled_buf`, with enables using the unfilled part of a cursor as a `BorrowedBuf`.
Note that unlike the ACP, `BorrowedCursor::unfilled_buf` was moved to a `From` conversion. This is more consistent with other ways of creating a `BorrowedBuf` and hides a bit this conversion that requires unsafe code to be used correctly.
Cc rust-lang/rust#78485rust-lang/rust#117693
Run bootstrap tests sooner in the `x test` pipeline
With the recently added bootstrap snapshot tests, and in general with our plans to test more things in bootstrap, I feel like the original comment isn't accurate anymore. Recently, on several occasions I had to wait for 40+ minutes of CI just to find out that the bootstrap snapshot tests have failed. I think we should run bootstrap tests towards the beginning instead now.
r? ```@jieyouxu```
Use zero for initialized Once state
By re-labeling which integer represents which internal state for `Once` we can ensure that the initialized state is the all-zero state. This is beneficial because some CPU architectures (such as Arm) have specialized instructions to specifically branch on non-zero, and checking for the initialized state is by far the most important operation.
As an example, take this:
```rust
use std::sync::atomic::{AtomicU32, Ordering};
const INIT: u32 = 3;
#[inline(never)]
#[cold]
pub fn slow(state: &AtomicU32) {
state.store(INIT, Ordering::Release);
}
pub fn ensure_init(state: &AtomicU32) {
if state.load(Ordering::Acquire) != INIT {
slow(state)
}
}
```
If `INIT` is 3 (as is currently the state for `Once`), we see the following assembly on `aarch64-apple-darwin`:
```asm
example::ensure_init::h332061368366e313:
ldapr w8, [x0]
cmp w8, #3
b.ne LBB1_2
ret
LBB1_2:
b example::slow::ha042bd6a4f33724e
```
By changing the `INIT` state to zero we get the following:
```asm
example::ensure_init::h332061368366e313:
ldapr w8, [x0]
cbnz w8, LBB1_2
ret
LBB1_2:
b example::slow::ha042bd6a4f33724e
```
So this PR saves 1 instruction every time a `LazyLock` gets accessed on platforms such as these.
bootstrap/miri: avoid rebuilds for test builds
When building Miri in its own repo, we always build with `--all-targets`:
a009612691/src/tools/miri/miri-script/src/util.rs (L167-L174)
This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do `cargo build && cargo test` (which is typically what you end up doing inside `./miri test` and also inside `./x test miri`).
This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)
make `cfg_select` a builtin macro
tracking issue: https://github.com/rust-lang/rust/issues/115585
This parses mostly the same as the `macro cfg_select` version, except:
1. wrapping in double brackets is no longer supported (or needed): `cfg_select {{ /* ... */ }}` is now rejected.
2. in an expression context, the rhs is no longer wrapped in a block, so that this now works:
```rust
fn main() {
println!(cfg_select! {
unix => { "foo" }
_ => { "bar" }
});
}
```
3. a single wildcard rule is now supported: `cfg_select { _ => 1 }` now works
I've also added an error if none of the rules evaluate to true, and warnings for any arms that follow the `_` wildcard rule.
cc `@traviscross` if I'm missing any feature that should/should not be included
r? `@petrochenkov` for the macro logic details
Run `tests/rustdoc-json/attrs/target_features` on all hosts.
Makes it more convenient to test rustdoc on non x86_64. I mainly care about the aarch64 dev-desktops.
This works because rustdoc uses all target features, not just that of the target.
de-duplicate condition scoping logic between AST→HIR lowering and `ScopeTree` construction
There was some overlap between `rustc_ast_lowering::LoweringContext::lower_cond` and `rustc_hir_analysis::check::region::resolve_expr`, so I've removed the former and migrated its logic to the latter, with some simplifications.
Consequences:
- For `while` and `if` expressions' `let`-chains, this changes the `HirId`s for the `&&`s to properly correspond to their AST nodes. This is how guards were handled already.
- This makes match guards share previously-duplicated logic with `if`/`while` expressions. This will also be used by guard pattern[^1] guards.
- Aside from legacy syntax extensions (e.g. some builtin macros) that directly feed AST to the compiler, it's currently impossible to put attributes directly on `&&` operators in `let` chains[^2]. Nonetheless, attributes on `&&` operators in `let` chains in `if`/`while` expression conditions are no longer silently ignored and will be lowered.
- This no longer wraps conditions in `DropTemps`, so the HIR and THIR will be slightly smaller.
- `DesugaringKind::CondTemporary` is now gone. It's no longer applied to any spans, and all uses of it were dead since they were made to account for `if` and `while` being desugared to `match` on a boolean scrutinee.
- Should be a marginal perf improvement beyond that due to leveraging [`ScopeTree` construction](5e749eb66f/compiler/rustc_hir_analysis/src/check/region.rs (L312-L355))'s clever handling of `&&` and `||`:
- This removes some unnecessary terminating scopes that were placed around top-level `&&` and `||` operators in conditions. When lowered to MIR, logical operator chains don't create intermediate boolean temporaries, so there's no temporary to drop. The linked snippet handles wrapping the operands in terminating scopes as necessary, in case they create temporaries.
- The linked snippet takes care of letting `let` temporaries live and terminating other operands, so we don't need separate traversals of `&&` chains for that.
[^1]: rust-lang/rust#129967
[^2]: Case-by-case, here's my justification: `#[attr] e1 && e2` applies the attribute to `e1`. In `#[attr] (e1 && e2)` , the attribute is on the parentheses in the AST, plus it'd fail to parse if `e1` or `e2` contains a `let`. In `#[attr] expands_to_let_chain!()`, the attribute would already be ignored (rust-lang/rust#63221) and it'd fail to parse anyway; even if the expansion site is a condition, the expansion wouldn't be parsed with `Restrictions::ALLOW_LET`. If it *was* allowed, the notion of a "reparse context" from https://github.com/rust-lang/rust/issues/61733#issuecomment-509626449 would be necessary in order to make `let`-chains left-associative; multiple places in the compiler assume they are.
Split up the `unknown_or_malformed_diagnostic_attributes` lint
This splits up the lint into the following lint group:
- `unknown_diagnostic_attributes` - triggers if the attribute is unknown to the current compiler
- `misplaced_diagnostic_attributes` - triggers if the attribute exists but it is not placed on the item kind it's meant for
- `malformed_diagnostic_attributes` - triggers if the attribute's syntax or options are invalid
- `malformed_diagnostic_format_literals` - triggers if the format string literal is invalid, for example if it has unpaired curly braces or invalid parameters
- this pr doesn't create it, but future lints for things like deprecations can also go here.
This PR does not start emitting lints in places that previously did not.
## Motivation
I want to have finer control over what `unknown_or_malformed_diagnostic_attributes` does
I have a project with fairly low msrv that is/will have a lower msrv than future diagnostic attributes. So lints will be emitted when I or others compile it on a lower msrv.
At this time, there are two options to silence these lints:
- `#[allow(unknown_or_malformed_diagnostic_attributes)]` - this risks diagnostic regressions if I (or others) mess up using the attribute, or if the attribute's syntax ever changes.
- write a build script to detect the compiler version and emit cfgs, and then conditionally enable the attribute:
```rust
#[cfg_attr(rust_version_99, diagnostic::new_attr_in_rust_99(thing = ..))]`
struct Foo;
```
or conditionally `allow` the lint:
```rust
// lib.rs
#![cfg_attr(not(current_rust), allow(unknown_or_malformed_diagnostic_attributes))]
```
I like to avoid using build scripts if I can, so the following works much better for me. That is what this PR will let me do in the future:
```rust
#[allow(unknown_diagnostic_attribute, reason = "attribute came out in rust 1.99 but msrv is 1.70")]
#[diagnostic::new_attr_in_rust_99(thing = ..)]`
struct Foo;
Rollup of 9 pull requests
Successful merges:
- rust-lang/rust#143403 (Port several trait/coherence-related attributes the new attribute system)
- rust-lang/rust#143633 (fix: correct assertion to check for 'noinline' attribute presence before removal)
- rust-lang/rust#143647 (Clarify and expand documentation for std::sys_common dependency structure)
- rust-lang/rust#143716 (compiler: doc/comment some codegen-for-functions interfaces)
- rust-lang/rust#143747 (Add target maintainer information for aarch64-unknown-linux-musl)
- rust-lang/rust#143759 (Fix typos in function names in the `target_feature` test)
- rust-lang/rust#143767 (Bump `src/tools/x` to Edition 2024 and some cleanups)
- rust-lang/rust#143769 (Remove support for SwitchInt edge effects in backward dataflow)
- rust-lang/rust#143770 (build-helper: clippy fixes)
r? `@ghost`
`@rustbot` modify labels: rollup
Clarify and expand documentation for std::sys_common dependency structure
This PR makes a minor improvement to the module-level documentation of std::sys_common:
Replaces the lowercase “dag” with the more standard and explicit form “DAG (Directed Acyclic Graph)” for clarity.
Port several trait/coherence-related attributes the new attribute system
Part of rust-lang/rust#131229
This ports:
- `#[const_trait]`
- `#[rustc_deny_explicit_impl]`
- `#[rustc_do_not_implement_via_object]`
- `#[rustc_coinductive]`
- `#[type_const]`
- `#[rustc_specialization_trait]`
- `#[rustc_unsafe_specialization_marker]`
- `#[marker]`
- `#[fundamental]`
- `#[rustc_paren_sugar]`
- `#[rustc_allow_incoherent_impl]`
- `#[rustc_coherence_is_core]`
This also changes `#[marker]` to error on duplicates instead of warning.
cc rust-lang/rust#142838, but I don't think it matters too much, since it's unstable.
r? ``@oli-obk``
Make UB transmutes really UB in LLVM
Ralf suggested in <https://github.com/rust-lang/rust/pull/143410#discussion_r2184928123> that UB transmutes shouldn't be trapping, which happened for the one path *that* PR was changing, but there's another path as well, so *this* PR changes that other path to match.
r? codegen
fix: Include frontmatter in -Zunpretty output
In the implementation (rust-lang/rust#140035), this was left as an open question for
the tracking issue (rust-lang/rust#136889). My assumption is that this should be
carried over.
The test was carried over from rust-lang/rust#137193 which was superseded by rust-lang/rust#140035.
Thankfully, either way, `-Zunpretty` is unstable and we can always
change it even if we stabilize frontmatter.
chore: Improve how the other suggestions message gets rendered
Note: This change is part of my ongoing work to use `annotate-snippets` as `rustc`'s emitter
This change started as a way to remove some specialty code paths from `annotate-snippets`, by making the "and {} other candidates" message get rendered like a secondary message with no level, but turned into a fix for the message's Unicode output. Before this change, when using the Unicode output, the other suggestions message would get rendered outside of the main suggestion block, making it feel disconnected from what it was referring to. This change makes it so that the message is on the last line of the block, aligning its rendering with other secondary messages, and making it clear what the message is referring to.
Before:
```
error[E0433]: failed to resolve: use of undeclared type `IntoIter`
╭▸ $DIR/issue-82956.rs:28:24
│
LL │ let mut iter = IntoIter::new(self);
│ ━━━━━━━━ use of undeclared type `IntoIter`
╰╴
help: consider importing one of these structs
╭╴
LL + use std::array::IntoIter;
├╴
LL + use std::collections::binary_heap::IntoIter;
├╴
LL + use std::collections::btree_map::IntoIter;
├╴
LL + use std::collections::btree_set::IntoIter;
╰╴
and 9 other candidates
```
After:
```
error[E0433]: failed to resolve: use of undeclared type `IntoIter`
╭▸ $DIR/issue-82956.rs:28:24
│
LL │ let mut iter = IntoIter::new(self);
│ ━━━━━━━━ use of undeclared type `IntoIter`
╰╴
help: consider importing one of these structs
╭╴
LL + use std::array::IntoIter;
├╴
LL + use std::collections::binary_heap::IntoIter;
├╴
LL + use std::collections::btree_map::IntoIter;
├╴
LL + use std::collections::btree_set::IntoIter;
│
╰ and 9 other candidates
```
`tests/ui`: A New Order [28/28] FINAL PART
> [!NOTE]
>
> Intermediate commits are intended to help review, but will be squashed prior to merge.
Some `tests/ui/` housekeeping, to trim down number of tests directly under `tests/ui/`. Part of rust-lang/rust#133895.
r? ``@tgross35``
`tests/ui`: A New Order [27/N]
> [!NOTE]
>
> Intermediate commits are intended to help review, but will be squashed prior to merge.
Some `tests/ui/` housekeeping, to trim down number of tests directly under `tests/ui/`. Part of rust-lang/rust#133895.
r? ``@tgross35``
rust: library: Add `setsid` method to `CommandExt` trait
Add a setsid method to the CommandExt trait so that callers can create a process in a new session and process group whilst still using the POSIX spawn fast path.
Tracking issue: rust-lang/rust#105376
ACP: https://github.com/rust-lang/libs-team/issues/184
This PR was previously submitted by ``@HarveyHunt`` (whom I marked as Co-Author in the commit message) in rust-lang/rust#105377. However that PR went stale.
I applied the [suggestion](231d19fcbf (r1893457943)) to change the function signature to `fn setsid(&mut self, setsid: bool) -> &mut Command`.
Add profiler to bootstrap command
This PR adds command profiling to the bootstrap command. It tracks the total execution time and records cache hits for each command. It also provides the ability to export execution result to a JSON file. Integrating this with Chrome tracing could further enhance observability.
r? `@Kobzol`
triagebot.toml: ping lolbinarycat if tidy extra checks were modified
I rewrote a large chunk of this module, and plan to do further changes to it (namely moving rustdoc_js checks into it), so it would be nice to keep up with and provide feedback on any changes to it, at least for the immediate future.
r? `@Kobzol`
Assorted `run-make-support` maintenance
This PR should contain no functional changes.
- Commit 1: Removes the support library's CHANGELOG. In the very beginning, I thought maybe we would try to version this library. But this is a purely internal test support library, and it's just extra busywork trying to maintain changelog/versions. It's also hopelessly outdated.
- Commit 2: Resets version number to `0.0.0`. Ditto on busywork.
- Commit 3: Bump `run-make-support` to Edition 2024. The support library was already "compliant" with Edition 2024.
- Commit 4: Slightly organizes the support library dependencies.
- Commit 5: Previously, I tried hopelessly to maintain some manual formatting, but that was annoying because it required skipping rustfmt (so export ordering etc. could not be extra formatted). Give up, and do some rearrangements / module prefix tricks to get the `lib.rs` looking at least *reasonable*. IMO this is not a strict improvement, but I rather regain the ability to auto-format it with rustfmt.
- Commit {6,7}: Noticed in rust-lang/rust#143669 that we apparently had *both* {`is_msvc`, `is_windows_msvc`}. This PR removes `is_msvc` in favor of `is_windows_msvc` to make it unambiguous (and only retain one way of gating) as there are some UEFI targets which are MSVC but not Windows.
Best reviewed commit-by-commit.
r? `@Kobzol`
Re-expose nested bodies in rustc_borrowck::consumers
After https://github.com/rust-lang/rust/pull/138499, it's not possible anymore to get borrowck information for nested bodies via `get_body_with_borrowck_facts`. This PR re-exposes nested bodies by returning a map containing the typeck root and all its nested bodies. To collect the bodies, a map is added to `BorrowCheckRootCtxt`, and a body is inserted every time `do_mir_borrowck` is called.
r? ``@lcnr``
