Make __rust_alloc_error_handler_should_panic a function
Fixesrust-lang/rust#143253
`__rust_alloc_error_handler_should_panic` is a static but was being exported as a function.
For most targets this doesn't matter, but Arm64EC Windows uses different decorations for exported variables vs functions, hence it fails to link when `-Z oom=abort` is enabled.
We've had issues in the past with statics like this (see rust-lang/rust#141061) but the tldr; is that Arm64EC needs symbols correctly exported as either a function or data, and data MUST and MUST ONLY be marked `dllimport` when the symbol is being imported from another binary, which is non-trivial to calculate for these compiler-generated statics.
So, instead, the easiest thing to do is to make `__rust_alloc_error_handler_should_panic` a function instead.
Since `__rust_alloc_error_handler_should_panic` isn't involved in any linking shenanigans, I've marked it as `AlwaysInline` with the hopes that the various backends will see that it is just returning a constant and perform the same optimizations as the previous implementation.
r? `@bjorn3`
Most uses of it either contain a fat or thin lto module. Only
WorkItem::LTO could contain both, but splitting that enum variant
doesn't complicate things much.
Disable f16 on Aarch64 without neon for llvm < 20.1.1
This check was added unconditionally in c51b229140 ("Disable f16 on Aarch64 without `neon`") and reverted in 4a8d35709e ("Revert "Disable `f16` on Aarch64 without `neon`"") since it did not fail in Rust's build. However, it is still possible to hit this crash if using LLVM 19 built with assertions, so disable the type conditionally based on version here.
Note that for these builds, a similar patch is needed in the build script for `compiler-builtins` since it does not yet use `cfg(target_has_reliable_f16)` (hopefully to be resolved in the near future).
Report: https://github.com/rust-lang/rust/pull/139276#issuecomment-3014781652
Original LLVM issue: https://github.com/llvm/llvm-project/issues/129394
give Pointer::into_parts a more scary name and offer a safer alternative
`into_parts` is a bit too innocent of a name for a somewhat subtle operation.
r? `@oli-obk`
This check was added unconditionally in c51b229140 ("Disable f16 on
Aarch64 without `neon`") and reverted in 4a8d35709e ("Revert "Disable
`f16` on Aarch64 without `neon`"") since it did not fail in Rust's
build. However, it is still possible to hit this crash if using LLVM 19
built with assertions, so disable the type conditionally based on
version here.
Note that for these builds, a similar patch is needed in the build
script for `compiler-builtins` since it does not yet use
`cfg(target_has_reliable_f16)` (hopefully to be resolved in the near
future).
Report: https://www.github.com/rust-lang/rust/pull/139276#issuecomment-3014781652
Original LLVM issue: https://www.github.com/llvm/llvm-project/issues/129394
Add SIMD funnel shift and round-to-even intrinsics
This PR adds 3 new SIMD intrinsics
- `simd_funnel_shl` - funnel shift left
- `simd_funnel_shr` - funnel shift right
- `simd_round_ties_even` (vector version of `round_ties_even_fN`)
TODO (future PR): implement `simd_fsh{l,r}` in miri, cg_gcc and cg_clif (it is surprisingly hard to implement without branches, the common tricks that rotate uses doesn't work because we have 2 elements now. e.g, the `-n&31` trick used by cg_gcc to implement rotate doesn't work with this because then `fshl(a, b, 0)` will be `a | b`)
[#t-compiler > More SIMD intrinsics](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/More.20SIMD.20intrinsics/with/522130286)
`@rustbot` label T-compiler T-libs A-intrinsics F-core_intrinsics
r? `@workingjubilee`
Rollup of 7 pull requests
Successful merges:
- rust-lang/rust#142146 (Withdraw the claim `extern "C-cmse-nonsecure-*"` always matches `extern "C"`)
- rust-lang/rust#142200 (`tests/ui`: A New Order [8/N])
- rust-lang/rust#142724 (Add runtime check to avoid overwrite arg in `Diag`)
- rust-lang/rust#142809 (Add PrintTAFn flag for targeted type analysis printing)
- rust-lang/rust#142976 (Check CoerceUnsized impl validity before coercing)
- rust-lang/rust#142992 (Convert some ABI tests to use `extern "rust-invalid"`)
- rust-lang/rust#143000 (Make `Sub`, `Mul`, `Div` and `Rem` `const_traits`)
r? `@ghost`
`@rustbot` modify labels: rollup
Add PrintTAFn flag for targeted type analysis printing
## Summary
This PR adds a new `PrintTAFn` flag to the `-Z autodiff` option that allows printing type analysis information for a specific function, rather than all functions.
## Changes
### New Flag
- Added `PrintTAFn=<function_name>` option to `-Z autodiff`
- Usage: `-Z autodiff=Enable,PrintTAFn=my_function_name`
### Implementation Details
- **Rust side**: Added `PrintTAFn(String)` variant to `AutoDiff` enum
- **Parser**: Updated `parse_autodiff` to handle `PrintTAFn=<function_name>` syntax with proper error handling
- **FFI**: Added `set_print_type_fun` function to interface with Enzyme's `FunctionToAnalyze` command line option
- **Documentation**: Updated help text and documentation for the new flag
### Files Modified
- `compiler/rustc_session/src/config.rs`: Added `PrintTAFn(String)` variant
- `compiler/rustc_session/src/options.rs`: Updated parser and help text (now shows `PrintTAFn` in the list)
- `compiler/rustc_codegen_llvm/src/llvm/enzyme_ffi.rs`: Added FFI function and static variable
- `compiler/rustc_codegen_llvm/src/back/lto.rs`: Added handling for new flag
- `src/doc/rustc-dev-guide/src/autodiff/flags.md`: Updated documentation
- `src/doc/unstable-book/src/compiler-flags/autodiff.md`: Updated documentation
## Testing
The flag can be tested with:
```bash
rustc +enzyme -Z autodiff=Enable,PrintTAFn=square test.rs
```
This will print type analysis information only for the function named "square" instead of all functions.
## Error Handling
The parser includes proper error handling:
- Missing argument: `PrintTAFn` without `=<function_name>` will show an error
- Unknown options: Invalid autodiff options will be reported
r? ```@ZuseZ4```
Remove dead instructions in terminate blocks
Terminate blocks look pretty in the IR I've looked at, so no actual perf delta from this. But it seems reasonable to note produce unused IR.
Normally LLVM and rustc agree about what features are implied by
target-cpu, but for NVPTX, LLVM considers sm_* and ptx* features to be
exclusive, which makes sense for codegen purposes. But in Rust, we want
to think of them as:
sm_{sver} means that the target supports the hardware features of sver
ptx{pver} means the driver supports PTX ISA pver
Intrinsics usually require a minimum sm_{sver} and ptx{pver}.
Prior to this commit, -Ctarget-cpu=sm_70 would activate only sm_70 and
ptx60 (the minimum PTX version that supports sm_70, which maximizes
driver compatibility). With this commit, it also activates all the
implied target features (sm_20, ..., sm_62; ptx32, ..., ptx50).
Add target features for sm_* and ptx*, both of which form a partial
order, but cannot be combined to a single partial order. These mirror
the LLVM target features, but we do not provide LLVM target
processors (which imply both an sm_* and ptx* feature).
Add some documentation for the nvptx target.
Change __rust_no_alloc_shim_is_unstable to be a function
This fixes a long sequence of issues:
1. A customer reported that building for Arm64EC was broken: #138541
2. This was caused by a bug in my original implementation of Arm64EC support, namely that only functions on Arm64EC need to be decorated with `#` but Rust was decorating statics as well.
3. Once I corrected Rust to only decorate functions, I started linking failures where the linker couldn't find statics exported by dylib dependencies. This was caused by the compiler not marking exported statics in the generated DEF file with `DATA`, thus they were being exported as functions not data.
4. Once I corrected the way that the DEF files were being emitted, the linker started failing saying that it couldn't find `__rust_no_alloc_shim_is_unstable`. This is because the MSVC linker requires the declarations of statics imported from other dylibs to be marked with `dllimport` (whereas it will happily link to functions imported from other dylibs whether they are marked `dllimport` or not).
5. I then made a change to ensure that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport`, but the MSVC linker started emitting warnings that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport` but was declared in an obj file. This is a harmless warning which is a performance hint: anything that's marked `dllimport` must be indirected via an `__imp` symbol so I added a linker arg in the target to suppress the warning.
6. A customer then reported a similar warning when using `lld-link` (<https://github.com/rust-lang/rust/pull/140176#issuecomment-2872448443>). I don't think it was an implementation difference between the two linkers but rather that, depending on the obj that the declaration versus uses of `__rust_no_alloc_shim_is_unstable` landed in we would get different warnings, so I suppressed that warning as well: #140954.
7. Another customer reported that they weren't using the Rust compiler to invoke the linker, thus these warnings were breaking their build: <https://github.com/rust-lang/rust/pull/140176#issuecomment-2881867433>. At that point, my original change was reverted (#141024) leaving Arm64EC broken yet again.
Taking a step back, a lot of these linker issues arise from the fact that `__rust_no_alloc_shim_is_unstable` is marked as `extern "Rust"` in the standard library and, therefore, assumed to be a foreign item from a different crate BUT the Rust compiler may choose to generate it either in the current crate, some other crate that will be statically linked in OR some other crate that will by dynamically imported.
Worse yet, it is impossible while building a given crate to know if `__rust_no_alloc_shim_is_unstable` will statically linked or dynamically imported: it might be that one of its dependent crates is the one with an allocator kind set and thus that crate (which is compiled later) will decide depending if it has any dylib dependencies or not to import `__rust_no_alloc_shim_is_unstable` or generate it. Thus, there is no way to know if the declaration of `__rust_no_alloc_shim_is_unstable` should be marked with `dllimport` or not.
There is a simple fix for all this: there is no reason `__rust_no_alloc_shim_is_unstable` must be a static. It needs to be some symbol that must be linked in; thus, it could easily be a function instead. As a function, there is no need to mark it as `dllimport` when dynamically imported which avoids the entire mess above.
There may be a perf hit for changing the `volatile load` to be a `tail call`, so I'm happy to change that part back (although I question what the codegen of a `volatile load` would look like, and if the backend is going to try to use load-acquire semantics).
Build with this change applied BEFORE #140176 was reverted to demonstrate that there are no linking issues with either MSVC or MinGW: <https://github.com/rust-lang/rust/actions/runs/15078657205>
Incidentally, I fixed `tests/run-make/no-alloc-shim` to work with MSVC as I needed it to be able to test locally (FYI for #128602)
r? `@bjorn3`
cc `@jieyouxu`
Use `LLVMIntrinsicGetDeclaration` to completely remove the hardcoded intrinsics list
Follow-up to rust-lang/rust#142259
This also needs a rustc-perf run, because `Intrinsic::getType` can be expensive
`@rustbot` label A-LLVM A-codegen T-compiler
r? `@workingjubilee`
cc `@nikic`
Move metadata object generation for dylibs to the linker code
This deduplicates some code between codegen backends and may in the future allow adding extra metadata that is only known at link time.
Prerequisite of https://github.com/rust-lang/rust/issues/96708.
Simplify implementation of Rust intrinsics by using type parameters in the cache
The current implementation of intrinsics have a lot of duplication to handle different overloads of overloaded LLVM intrinsic. This PR uses the **base name and the type parameters** in the cache instead of the full, overloaded name. This has the benefit that `call_intrinsic` doesn't need to provide the full name, rather the type parameters (which is most of the time more available). This uses `LLVMIntrinsicCopyOverloadedName2` to get the overloaded name from the base name and the type parameters, and only uses it to declare the function.
(originally was part of rust-lang/rust#140763, split off later)
`@rustbot` label A-codegen A-LLVM
r? codegen
add `extern "custom"` functions
tracking issue: rust-lang/rust#140829
previous discussion: https://github.com/rust-lang/rust/issues/140566
In short, an `extern "custom"` function is a function with a custom ABI, that rust does not know about. Therefore, such functions can only be defined with `#[unsafe(naked)]` and `naked_asm!`, or via an `extern "C" { /* ... */ }` block. These functions cannot be called using normal rust syntax: calling them can only be done from inline assembly.
The motivation is low-level scenarios where a custom calling convention is used. Currently, we often pick `extern "C"`, but that is a lie because the function does not actually respect the C calling convention.
At the moment `"custom"` seems to be the name with the most support. That name is not final, but we need to pick something to actually implement this.
r? `@traviscross`
cc `@tgross35`
try-job: x86_64-apple-2
retpoline and retpoline-external-thunk flags (target modifiers) to enable retpoline-related target features
`-Zretpoline` and `-Zretpoline-external-thunk` flags are target modifiers (tracked to be equal in linked crates).
* Enables target features for `-Zretpoline-external-thunk`:
`+retpoline-external-thunk`, `+retpoline-indirect-branches`, `+retpoline-indirect-calls`.
* Enables target features for `-Zretpoline`:
`+retpoline-indirect-branches`, `+retpoline-indirect-calls`.
It corresponds to clang -mretpoline & -mretpoline-external-thunk flags.
Also this PR forbids to specify those target features manually (warning).
Issue: rust-lang/rust#116852
store `target.min_global_align` as an `Align`
Parse the alignment properly when the target is defined/parsed, and error out on invalid alignment values. That means this work doesn't need to happen for every global in each backend.
