Ignore coroutine witness type region args in auto trait confirmation
## The problem
Consider code like:
```
async fn process<'a>() {
Box::pin(process()).await;
}
fn require_send(_: impl Send) {}
fn main() {
require_send(process());
}
```
When proving that the coroutine `{coroutine@process}::<'?0>: Send`, we end up instantiating a nested goal `{witness@process}::<'?0>: Send` by synthesizing a witness type from the coroutine's args:
Proving a coroutine witness type implements an auto trait requires looking up the coroutine's witness types. The witness types are a binder that look like `for<'r> { Pin<Box<{coroutine@process}::<'r>>> }`. We instantiate this binder with placeholders and prove `Send` on the witness types. This ends up eventually needing to prove something like `{coroutine@process}::<'!1>: Send`. Repeat this process, and we end up in an overflow during fulfillment, since fulfillment does not use freshening.
This can be visualized with a trait stack that ends up looking like:
* `{coroutine@process}::<'?0>: Send`
* `{witness@process}::<'?0>: Send`
* `Pin<Box<{coroutine@process}::<'!1>>>: Send`
* `{coroutine@process}::<'!1>: Send`
* ...
* `{coroutine@process}::<'!2>: Send`
* `{witness@process}::<'!2>: Send`
* ...
* overflow!
The problem here specifically comes from the first step: synthesizing a witness type from the coroutine's args.
## Why wasn't this an issue before?
Specifically, before 63f6845e570305a92eaf855897768617366164d6, this wasn't an issue because we were instead extracting the witness from the coroutine type itself. It turns out that given some `{coroutine@process}::<'?0>`, the witness type was actually something like `{witness@process}::<'erased>`!
So why do we end up with a witness type with `'erased` in its args? This is due to the fact that opaque type inference erases all regions from the witness. This is actually explicitly part of opaque type inference -- changing this to actually visit the witness types actually replicates this overflow even with 63f6845e570305a92eaf855897768617366164d6 reverted:
ca77504943/compiler/rustc_borrowck/src/type_check/opaque_types.rs (L303-L313)
To better understand this difference and how it avoids a cycle, if you look at the trait stack before 63f6845e570305a92eaf855897768617366164d6, we end up with something like:
* `{coroutine@process}::<'?0>: Send`
* `{witness@process}::<'erased>: Send` **<-- THIS CHANGED**
* `Pin<Box<{coroutine@process}::<'!1>>>: Send`
* `{coroutine@process}::<'!1>: Send`
* ...
* `{coroutine@process}::<'erased>: Send` **<-- THIS CHANGED**
* `{witness@process}::<'erased>: Send` **<-- THIS CHANGED**
* coinductive cycle! 🎉
## So what's the fix?
This hack replicates the behavior in opaque type inference to erase regions from the witness type, but instead erasing the regions during auto trait confirmation. This is kinda a hack, but is sound. It does not need to be replicated in the new trait solver, of course.
---
I hope this explanation makes sense.
We could beta backport this instead of the revert https://github.com/rust-lang/rust/pull/145193, but then I'd like to un-revert that on master in this PR along with landing this this hack. Thoughts?
r? lcnr
Check coroutine upvars in dtorck constraint
Fixrust-lang/rust#144155.
This PR fixes an unsoundness where we were not considering coroutine upvars as drop-live if the coroutine interior types (witness types) had nothing which required drop.
In the case that the coroutine does not have any interior types that need to be dropped, then we don't need to treat all of the upvars as use-live; instead, this PR uses the same logic as closures, and descends into the upvar types to collect anything that must be drop-live. The rest of this PR is reworking the comment to explain the behavior here.
r? `@lcnr` or reassign 😸
---
Just some thoughts --- a proper fix for this whole situation would be to consider `TypingMode` in the `needs_drop` function, and just calling `coroutine_ty.needs_drop(tcx, typing_env)` in the dtorck constraint check.
During MIR building, we should probably use a typing mode that stalls the local coroutines and considers them to be unconditionally drop, or perhaps just stall *all* coroutines in analysis mode. Then in borrowck mode, we can re-check `needs_drop` but descend into witness types properly. https://github.com/rust-lang/rust/pull/144158 implements this experimentally.
This is a pretty involved fix, and conflicts with some in-flight changes (rust-lang/rust#144157) that I have around removing coroutine witnesses altogether. I'm happy to add a FIXME to rework this whole approach, but I don't want to block this quick fix since it's obviously more correct than the status-quo.
`TyCtxt::short_string` ensures that user visible type paths aren't overwhelming on the terminal output, and properly saves the long name to disk as a side-channel. We already use these throughout the compiler and have been using them as needed when users find cases where the output is verbose. This is a proactive search of some cases to use `short_string`.
We add support for shortening the path of "trait path only".
Every manual use of `short_string` is a bright marker that that error should be using structured diagnostics instead (as they have proper handling of long types without the maintainer having to think abou tthem).
When we don't actually print out a shortened type we don't need the "use `--verbose`" note.
On E0599 show type identity to avoid expanding the receiver's generic parameters.
Unify wording on `long_ty_path` everywhere.
`Printer` cleanups
The trait `Printer` is implemented by six types, and the sub-trait `PrettyPrinter` is implemented by three of those types. The traits and the impls are complex and a bit of a mess. This PR starts to clean them up.
r? ``@davidtwco``
Distinguish prepending and replacing self ty in predicates
There are two kinds of functions called `with_self_ty`:
1. Prepends the `Self` type onto an `ExistentialPredicate` which lacks it in its internal representation.
2. Replaces the `Self` type of an existing predicate, either for diagnostics purposes or in the new trait solver when normalizing that self type.
This PR distinguishes these two because I often want to only grep for one of them. Namely, let's call it `with_replaced_self_ty` when all we're doing is replacing the self type.
Currently they are mostly named `cx`, which is a terrible name for a
type that impls `Printer`/`PrettyPrinter`, and is easy to confuse with
other types like `TyCtxt`. This commit changes them to `p`. A couple of
existing `p` variables had to be renamed to make way.
This helps me understand the structure of the code a lot.
If any of these are actually reachable, we can put the old code back,
add a new test case, and we will have improved our test coverage.
Remove the witness type from coroutine *args* (without actually removing the type)
This does as much of rust-lang/rust#144157 as we can without having to break rust-lang/rust#143545 and/or introduce some better way of handling higher ranked assumptions.
Namely, it:
* Stalls coroutines based off of the *coroutine* type rather than the witness type.
* Reworks the dtorck constraint hack to not rely on the witness type.
* Removes the witness type from the args of the coroutine, eagerly creating the type for nested obligations when needed (auto/clone impls).
I'll experiment with actually removing the witness type in a follow-up.
r? lcnr
This commit changes it to store a `Region` instead of a `RegionVid` for the `Var` cases:
- We avoid having to call `Region::new_var` to re-create `Region`s from
`RegionVid`s in a few places, avoiding the interning process, giving a
small perf win. (At the cost of the type allowing some invalid
combinations of values.)
- All the cases now store two `Region`s, so the commit also separates
the `ConstraintKind` (a new type) from the `sub` and `sup` arguments
in `Constraint`.
Currently there is `Ty` and `BoundTy`, and `Region` and `BoundRegion`,
and `Const` and... `BoundVar`. An annoying inconsistency.
This commit repurposes the existing `BoundConst`, which was barely used,
so it's the partner to `Const`. Unlike `BoundTy`/`BoundRegion` it lacks
a `kind` field but it's still nice to have because it makes the const
code more similar to the ty/region code everywhere.
The commit also removes `impl From<BoundVar> for BoundTy`, which has a
single use and doesn't seem worth it.
These changes fix the "FIXME: We really should have a separate
`BoundConst` for consts".
uniquify root goals during HIR typeck
We need to rely on region identity to deal with hangs such as https://github.com/rust-lang/trait-system-refactor-initiative/issues/210 and to keep the current behavior of `fn try_merge_responses`.
This is a problem as borrowck starts by replacing each *occurrence* of a region with a unique inference variable. This frequently splits a single region during HIR typeck into multiple distinct regions. As we assume goals to always succeed during borrowck, relying on two occurances of a region being identical during HIR typeck causes ICE. See the now fixed examples in https://github.com/rust-lang/trait-system-refactor-initiative/issues/27 and rust-lang/rust#139409.
We've previously tried to avoid this issue by always *uniquifying* regions when canonicalizing goals. This prevents caching subtrees during canonicalization which resulted in hangs for very large types. People rely on such types in practice, which caused us to revert our attempt to reinstate `#[type_length_limit]` in https://github.com/rust-lang/rust/pull/127670. The complete list of changes here:
- rust-lang/rust#107981
- rust-lang/rust#110180
- rust-lang/rust#114117
- rust-lang/rust#130821
After more consideration, all occurrences of such large types need to happen outside of typeck/borrowck. We know this as we already walk over all types in the MIR body when replacing their regions with nll vars.
This PR therefore enables us to rely on region identity inside of the trait solver by exclusively **uniquifying root goals during HIR typeck**. These are the only goals we assume to hold during borrowck. This is insufficient as type inference variables may "hide" regions we later uniquify. Because of this, we now stash proven goals which depend on inference variables in HIR typeck and reprove them after writeback. This closes https://github.com/rust-lang/trait-system-refactor-initiative/issues/127.
This was originally part of rust-lang/rust#144258 but I've moved it into a separate PR. While I believe we need to rely on region identity to fix the performance issues in some way, I don't know whether rust-lang/rust#144258 is the best approach to actually do so. Regardless of how we deal with the hangs however, this change is necessary and desirable regardless.
r? `@compiler-errors` or `@BoxyUwU`
`-Zhigher-ranked-assumptions`: Consider WF of coroutine witness when proving outlives assumptions
### TL;DR
This PR introduces an unstable flag `-Zhigher-ranked-assumptions` which tests out a new algorithm for dealing with some of the higher-ranked outlives problems that come from auto trait bounds on coroutines. See:
* rust-lang/rust#110338
While it doesn't fix all of the issues, it certainly fixed many of them, so I'd like to get this landed so people can test the flag on their own code.
### Background
Consider, for example:
```rust
use std::future::Future;
trait Client {
type Connecting<'a>: Future + Send
where
Self: 'a;
fn connect(&self) -> Self::Connecting<'_>;
}
fn call_connect<C>(c: C) -> impl Future + Send
where
C: Client + Send + Sync,
{
async move { c.connect().await }
}
```
Due to the fact that we erase the lifetimes in a coroutine, we can think of the interior type of the async block as something like: `exists<'r, 's> { C, &'r C, C::Connecting<'s> }`. The first field is the `c` we capture, the second is the auto-ref that we perform on the call to `.connect()`, and the third is the resulting future we're awaiting at the first and only await point. Note that every region is uniquified differently in the interior types.
For the async block to be `Send`, we must prove that both of the interior types are `Send`. First, we have an `exists<'r, 's>` binder, which needs to be instantiated universally since we treat the regions in this binder as *unknown*[^exist]. This gives us two types: `{ &'!r C, C::Connecting<'!s> }`. Proving `&'!r C: Send` is easy due to a [`Send`](https://doc.rust-lang.org/nightly/std/marker/trait.Send.html#impl-Send-for-%26T) impl for references.
Proving `C::Connecting<'!s>: Send` can only be done via the item bound, which then requires `C: '!s` to hold (due to the `where Self: 'a` on the associated type definition). Unfortunately, we don't know that `C: '!s` since we stripped away any relationship between the interior type and the param `C`. This leads to a bogus borrow checker error today!
### Approach
Coroutine interiors are well-formed by virtue of them being borrow-checked, as long as their callers are invoking their parent functions in a well-formed way, then substitutions should also be well-formed. Therefore, in our example above, we should be able to deduce the assumption that `C: '!s` holds from the well-formedness of the interior type `C::Connecting<'!s>`.
This PR introduces the notion of *coroutine assumptions*, which are the outlives assumptions that we can assume hold due to the well-formedness of a coroutine's interior types. These are computed alongside the coroutine types in the `CoroutineWitnessTypes` struct. When we instantiate the binder when proving an auto trait for a coroutine, we instantiate the `CoroutineWitnessTypes` and stash these newly instantiated assumptions in the region storage in the `InferCtxt`. Later on in lexical region resolution or MIR borrowck, we use these registered assumptions to discharge any placeholder outlives obligations that we would otherwise not be able to prove.
### How well does it work?
I've added a ton of tests of different reported situations that users have shared on issues like rust-lang/rust#110338, and an (anecdotally) large number of those examples end up working straight out of the box! Some limitations are described below.
### How badly does it not work?
The behavior today is quite rudimentary, since we currently discharge the placeholder assumptions pretty early in region resolution. This manifests itself as some limitations on the code that we accept.
For example, `tests/ui/async-await/higher-ranked-auto-trait-11.rs` continues to fail. In that test, we must prove that a placeholder is equal to a universal for a param-env candidate to hold when proving an auto trait, e.g. `'!1 = 'a` is required to prove `T: Trait<'!1>` in a param-env that has `T: Trait<'a>`. Unfortunately, at that point in the MIR body, we only know that the placeholder is equal to some body-local existential NLL var `'?2`, which only gets equated to the universal `'a` when being stored into the return local later on in MIR borrowck.
This could be fixed by integrating these assumptions into the type outlives machinery in a more first-class way, and delaying things to the end of MIR typeck when we know the full relationship between existential and universal NLL vars. Doing this integration today is quite difficult today.
`tests/ui/async-await/higher-ranked-auto-trait-11.rs` fails because we don't compute the full transitive outlives relations between placeholders. In that test, we have in our region assumptions that some `'!1 = '!2` and `'!2 = '!3`, but we must prove `'!1 = '!3`.
This can be fixed by computing the set of coroutine outlives assumptions in a more transitive way, or as I mentioned above, integrating these assumptions into the type outlives machinery in a more first-class way, since it's already responsible for the transitive outlives assumptions of universals.
### Moving forward
I'm still quite happy with this implementation, and I'd like to land it for testing. I may work on overhauling both the way we compute these coroutine assumptions and also how we deal with the assumptions during (lexical/nll) region checking. But for now, I'd like to give users a chance to try out this new `-Zhigher-ranked-assumptions` flag to uncover more shortcomings.
[^exist]: Instantiating this binder with infer regions would be incomplete, since we'd be asking for *some* instantiation of the interior types, not proving something for *all* instantiations of the interior types.
Unify `CoroutineWitness` sooner in typeck, and stall coroutine obligations based off of `TypingEnv`
* Stall coroutine obligations based off of `TypingMode` in the old solver.
* Eagerly assign `TyKind::CoroutineWitness` to the witness arg of coroutines during typeck, rather than deferring them to the end of typeck.
r? lcnr
This is part of https://github.com/rust-lang/rust/issues/143017.
Use relative visibility when noting sealed trait to reduce false positive
Fixesrust-lang/rust#143392
I used relative visibility instead of just determining if it's public or not.
r? compiler
