Despite that the `fflags` register (representing floating point
exception flags) is stated as a flag register in the reference, it's not
in the default clobber list of the RISC-V inline assembly and it would
be better to fix it.
match clang's `va_arg` assembly on arm targets
tracking issue: https://github.com/rust-lang/rust/issues/44930
For this example
```rust
#![feature(c_variadic)]
#[unsafe(no_mangle)]
unsafe extern "C" fn variadic(a: f64, mut args: ...) -> f64 {
let b = args.arg::<f64>();
let c = args.arg::<f64>();
a + b + c
}
```
We currently generate (via llvm):
```asm
variadic:
sub sp, sp, #12
stmib sp, {r2, r3}
vmov d0, r0, r1
add r0, sp, #4
vldr d1, [sp, #4]
add r0, r0, #15
bic r0, r0, #7
vadd.f64 d0, d0, d1
add r1, r0, #8
str r1, [sp]
vldr d1, [r0]
vadd.f64 d0, d0, d1
vmov r0, r1, d0
add sp, sp, #12
bx lr
```
LLVM is not doing a good job. In fact, it's well-known that LLVM's implementation of `va_arg` is kind of bad, and we implement it ourselves (based on clang) for many targets already. For arm, our own `emit_ptr_va_arg` saves 3 instructions.
Next, it turns out it's important for LLVM to explicitly start and end the lifetime of the `va_list`. In https://github.com/rust-lang/rust/pull/146059 I already end the lifetime, but when looking at this again, I noticed that it is important to also start it, see https://godbolt.org/z/EGqvKTTsK: failing to explicitly start the lifetime uses an extra register.
So, the combination of `emit_ptr_va_arg` with starting/ending the lifetime makes rustc emit exactly the instructions that clang generates::
```asm
variadic:
sub sp, sp, #12
stmib sp, {r2, r3}
vmov d16, r0, r1
vldr d17, [sp, #4]
vadd.f64 d16, d16, d17
vldr d17, [sp, #12]
vadd.f64 d16, d16, d17
vmov r0, r1, d16
add sp, sp, #12
bx lr
```
The arguments to `emit_ptr_va_arg` are based on [the clang implementation](03dc2a41f3/clang/lib/CodeGen/Targets/ARM.cpp (L798-L844)).
r? ``@workingjubilee`` (I can re-roll if your queue is too full, but you do seem like the right person here)
try-job: armhf-gnu
Implement `#[rustc_align_static(N)]` on `static`s
Tracking issue: https://github.com/rust-lang/rust/issues/146177
```rust
#![feature(static_align)]
#[rustc_align_static(64)]
static SO_ALIGNED: u64 = 0;
```
We need a different attribute than `rustc_align` because unstable attributes are tied to their feature (we can't have two unstable features use the same unstable attribute). Otherwise this uses all of the same infrastructure as `#[rustc_align]`.
r? `@traviscross`
We need a different attribute than `rustc_align` because unstable attributes are
tied to their feature (we can't have two unstable features use the same
unstable attribute). Otherwise this uses all of the same infrastructure
as `#[rustc_align]`.
rename erase_regions to erase_and_anonymize_regions
I find it consistently confusing that `erase_regions` does more than replacing regions with `'erased`. it also makes some code look real goofy to be writing manual folders to erase regions with a comment saying "we cant use erase regions" :> or code that re-calls erase_regions on types with regions already erased just to anonymize all the bound regions.
r? lcnr
idk how i feel about the name being almost twice as long now
compiler: Include span of too huge array with `-Cdebuginfo=2`
We have a few ui tests to ensure we emit an error if we encounter too big arrays. Before this fix, compiling the tests with `-Cdebuginfo=2` would not include the spans of the instantiation sites, because the error is then emitted from a different code path that does not include the span.
Propagate the span to the error also in the debuginfo case, so the tests passes regardless of debuginfo level.
r? ``@wesleywiser`` since this is a natural continuation of https://github.com/rust-lang/rust/pull/145967 that you approved (thanks!).
cc https://github.com/rust-lang/rust/issues/61117 since this takes is one step closer to increasing `rust.debuginfo-level-tests` to `2` in the **x86_64-gnu-debug** CI job.
## Test failure output without the fix
<details>
<summary>
Here is what the test failures look like if you run the tests without the fix. (Click to expand.)
</summary>
```
$ ./x test --set rust.debuginfo-level-tests=2 tests/ui/limits/huge-array-simple-64.rs tests/ui/limits/huge-array.rs tests/ui/limits/issue-15919-64.rs
Building bootstrap
Finished `dev` profile [unoptimized] target(s) in 0.16s
/home/martin/src/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/llvm-strip does not exist; skipping copy
Building stage1 compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Finished `release` profile [optimized + debuginfo] target(s) in 0.40s
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Building stage1 lld-wrapper (stage0 -> stage1, x86_64-unknown-linux-gnu)
Finished `release` profile [optimized + debuginfo] target(s) in 0.08s
Building stage1 library artifacts (stage1 -> stage1, x86_64-unknown-linux-gnu)
Compiling addr2line v0.25.0
Compiling std v0.0.0 (/home/martin/src/rust/library/std)
Compiling rustc-std-workspace-std v1.99.0 (/home/martin/src/rust/library/rustc-std-workspace-std)
Compiling unicode-width v0.2.1
Compiling rustc-literal-escaper v0.0.5
Compiling proc_macro v0.0.0 (/home/martin/src/rust/library/proc_macro)
Compiling getopts v0.2.23
Compiling test v0.0.0 (/home/martin/src/rust/library/test)
Compiling sysroot v0.0.0 (/home/martin/src/rust/library/sysroot)
Finished `release` profile [optimized + debuginfo] target(s) in 14.43s
Building stage1 compiletest (stage0 -> stage1, x86_64-unknown-linux-gnu)
Finished `release` profile [optimized + debuginfo] target(s) in 0.14s
Testing stage2 compiletest suite=ui mode=ui (stage1 -> stage2, x86_64-unknown-linux-gnu)
running 6 tests
[ui] tests/ui/limits/huge-array.rs#full-debuginfo ... F
[ui] tests/ui/limits/issue-15919-64.rs#full-debuginfo ... F
[ui] tests/ui/limits/huge-array-simple-64.rs#full-debuginfo ... F
...
failures:
---- [ui] tests/ui/limits/huge-array.rs#full-debuginfo stdout ----
Saved the actual stderr to `/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/huge-array.full-debuginfo/huge-array.full-debuginfo.stderr`
diff of stderr:
1 error: values of the type `[[u8; 1518599999]; 1518600000]` are too big for the target architecture
- --> $DIR/huge-array.rs:9:9
- |
- LL | let s: [T; 1518600000] = [t; 1518600000];
- | ^
6
7 error: aborting due to 1 previous error
8
The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args limits/huge-array.rs`
error in revision `full-debuginfo`: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc" "/home/martin/src/rust/tests/ui/limits/huge-array.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/martin/.cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/martin/src/rust/vendor" "--sysroot" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/stage1" "--target=x86_64-unknown-linux-gnu" "--cfg" "full_debuginfo" "--check-cfg" "cfg(test,FALSE,no_debuginfo,full_debuginfo)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "--out-dir" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/huge-array.full-debuginfo" "-A" "unused" "-A" "internal_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Lnative=/home/martin/src/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Cdebuginfo=2"
stdout: none
--- stderr -------------------------------
error: values of the type `[[u8; 1518599999]; 1518600000]` are too big for the target architecture
error: aborting due to 1 previous error
------------------------------------------
---- [ui] tests/ui/limits/huge-array.rs#full-debuginfo stdout end ----
---- [ui] tests/ui/limits/issue-15919-64.rs#full-debuginfo stdout ----
Saved the actual stderr to `/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/issue-15919-64.full-debuginfo/issue-15919-64.full-debuginfo.stderr`
diff of stderr:
1 error: values of the type `[usize; usize::MAX]` are too big for the target architecture
- --> $DIR/issue-15919-64.rs:10:9
- |
- LL | let x = [0usize; 0xffff_ffff_ffff_ffff];
- | ^
6
7 error: aborting due to 1 previous error
8
The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args limits/issue-15919-64.rs`
error in revision `full-debuginfo`: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc" "/home/martin/src/rust/tests/ui/limits/issue-15919-64.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/martin/.cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/martin/src/rust/vendor" "--sysroot" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/stage1" "--target=x86_64-unknown-linux-gnu" "--cfg" "full_debuginfo" "--check-cfg" "cfg(test,FALSE,no_debuginfo,full_debuginfo)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "--out-dir" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/issue-15919-64.full-debuginfo" "-A" "unused" "-A" "internal_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Lnative=/home/martin/src/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Cdebuginfo=2"
stdout: none
--- stderr -------------------------------
error: values of the type `[usize; usize::MAX]` are too big for the target architecture
error: aborting due to 1 previous error
------------------------------------------
---- [ui] tests/ui/limits/issue-15919-64.rs#full-debuginfo stdout end ----
---- [ui] tests/ui/limits/huge-array-simple-64.rs#full-debuginfo stdout ----
Saved the actual stderr to `/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/huge-array-simple-64.full-debuginfo/huge-array-simple-64.full-debuginfo.stderr`
diff of stderr:
1 error: values of the type `[u8; 2305843011361177600]` are too big for the target architecture
- --> $DIR/huge-array-simple-64.rs:12:9
- |
- LL | let _fat: [u8; (1<<61)+(1<<31)] =
- | ^^^^
6
7 error: aborting due to 1 previous error
8
The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args limits/huge-array-simple-64.rs`
error in revision `full-debuginfo`: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc" "/home/martin/src/rust/tests/ui/limits/huge-array-simple-64.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/martin/.cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/martin/src/rust/vendor" "--sysroot" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/stage1" "--target=x86_64-unknown-linux-gnu" "--cfg" "full_debuginfo" "--check-cfg" "cfg(test,FALSE,no_debuginfo,full_debuginfo)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "--out-dir" "/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/huge-array-simple-64.full-debuginfo" "-A" "unused" "-A" "internal_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Lnative=/home/martin/src/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Cdebuginfo=2"
stdout: none
--- stderr -------------------------------
error: values of the type `[u8; 2305843011361177600]` are too big for the target architecture
error: aborting due to 1 previous error
------------------------------------------
---- [ui] tests/ui/limits/huge-array-simple-64.rs#full-debuginfo stdout end ----
failures:
[ui] tests/ui/limits/huge-array.rs#full-debuginfo
[ui] tests/ui/limits/issue-15919-64.rs#full-debuginfo
[ui] tests/ui/limits/huge-array-simple-64.rs#full-debuginfo
test result: FAILED. 3 passed; 3 failed; 0 ignored; 0 measured; 19720 filtered out; finished in 117.18ms
Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
Build completed unsuccessfully in 0:00:17
```
</details>
As can be seen, the span info is missing with debuginfo=2 without the fix.
gpu offload: change suspicious map into filter
Fixes clippy warning:
```text
warning: this call to `map()` won't have an effect on the call to `count()`
--> compiler/rustc_codegen_llvm/src/builder/gpu_offload.rs:194:25
|
194 | let num_ptr_types = types
| _________________________^
195 | | .iter()
196 | | .map(|&x| matches!(cx.type_kind(x), rustc_codegen_ssa::common::TypeKind::Pointer))
197 | | .count();
| |________________^
|
= help: make sure you did not confuse `map` with `filter`, `for_each` or `inspect`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#suspicious_map
= note: `-W clippy::suspicious-map` implied by `-W clippy::suspicious`
= help: to override `-W clippy::suspicious` add `#[allow(clippy::suspicious_map)]`
```
Rework how the codegen coordinator code handles the allocator shim
Continuing from https://github.com/rust-lang/rust/pull/144503 this centralizes most handling of the allocator shim to a single 4 line block in the codegen coordinator. The allocator shim is small enough that making it go through the main codegen loop and spawning a worker thread for it is wasted effort.
A lot of places had special handling just in case they would get an
allocator module even though most of these places could never get one or
would have a trivial implementation for the allocator module. Moving all
handling of the allocator module to a single place simplifies things a
fair bit.
Implement Integer funnel shifts
Tracking issue: rust-lang/rust#145686
ACP: https://github.com/rust-lang/libs-team/issues/642
This implements funnel shifts on primitive integer types. Implements this for cg_llvm, with a fallback impl for everything else
Thanks `@folkertdev` for the fixes and tests
cc `@rust-lang/libs-api`
Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.
Rather than adding the inline always attribute to the function definition, we add it to the callsite. We can then check that the target features match and that the call would be safe to inline. If the function isn't inlined due to a mismatch, we emit a warning informing the user that the function can't be inlined due to the target feature mismatch.
See tracking issue rust-lang/rust#145574
This was done in #145740 and #145947. It is causing problems for people
using r-a on anything that uses the rustc-dev rustup package, e.g. Miri,
clippy.
This repository has lots of submodules and subtrees and various
different projects are carved out of pieces of it. It seems like
`[workspace.dependencies]` will just be more trouble than it's worth.
explicitly end the lifetime of `va_list`
tracking issue: https://github.com/rust-lang/rust/issues/44930
split out from: https://github.com/rust-lang/rust/pull/144549
The `va_list` is created in the compiler itself when the variable argument list `...` is desugared, and hence the lifetime end is not inserted automatically. The value can't outlive the function in which it was created, so it is correct to end the lifetime here. Ending the lifetime explicitly also appears to give slightly better codegen in https://github.com/rust-lang/rust/pull/144549.
I also included a little drive-by improvement to not cast pointers to integers and back again.
r? codegen
Update to ar_archive_writer 0.5
This updates `ar_archive_writer` to 0.5, which in turn was updated to match LLVM 20.1.8: <https://github.com/rust-lang/ar_archive_writer/pull/24>
As part of this, I refactored part of `SymbolWrapper.cpp` to pull common code that I was about to duplicate again into a new function.
NOTE: `ar_archive_writer` does include a breaking change where it no longer supports mangling C++ mangled names for Arm64EC. Since we don't need the mangled name (it's not the "exported name" which we're trying to load from the external dll), I'm setting the `import_name` when building for Arm64EC to prevent error when failing to mangle.
r? `@bjorn3`
We have a few ui tests to ensure we emit an error if we encounter too
big arrays. Before this fix, compiling the tests with `-Cdebuginfo=2`
would not include the spans of the instantiation sites, because the
error is then emitted from a different code path that does not include
the span.
Propagate the span to the error also in the debuginfo case, so the tests
passes regardless of debuginfo level.
compiler: Include span of too huge enum with `-Cdebuginfo=2`
We have the ui test `tests/ui/limits/huge-enum.rs` to ensure we emit an error if we encounter too big enums. Before this fix, compiling the test with `-Cdebuginfo=2` would not include the span of the instantiation site, because the error is then emitted from a different code path that does not include the span.
Propagate the span to the error also in the debuginfo case, so the test passes regardless of debuginfo level. I'm sure we can propagate spans in more places, but let's start small.
## Test failure without the fix
Here is what the failure looks like if you run the test without the fix:
```
[ui] tests/ui/limits/huge-enum.rs#full-debuginfo ... F
.
failures:
---- [ui] tests/ui/limits/huge-enum.rs#full-debuginfo stdout ----
Saved the actual stderr to `/home/martin/src/rust/build/x86_64-unknown-linux-gnu/test/ui/limits/huge-enum.full-debuginfo/huge-enum.full-debuginfo.stderr`
diff of stderr:
1 error: values of the type `Option<TYPE>` are too big for the target architecture
- --> $DIR/huge-enum.rs:17:9
- |
- LL | let big: BIG = None;
- | ^^^
6
7 error: aborting due to 1 previous error
8
The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args limits/huge-enum.rs`
```
as can be seen, the `span` used to be missing with `debuginfo=2`.
## See also
This is one small step towards resolving rust-lang/rust#61117.
cc https://github.com/rust-lang/rust/pull/144499 which began running UI tests with `rust.debuginfo-level-tests=1`. This PR is part of preparing for increasing that to debuglevel 2.
Don't export them from cdylibs. There is no need to do so and it
complicates exported_non_generic_symbols. In addition the GCC backend
likely uses different symbols and may potentially not even need us to
explicitly tell it to export the symbols it needs.
We have a ui test to ensure we emit an error if we encounter too big
enums. Before this fix, compiling the test with `-Cdebuginfo=2` would
not include the span of the instantiation site, because the error is
then emitted from a different code path that does not include the span.
Propagate the span to the error also in the debuginfo case, so the test
passes regardless of debuginfo level.
Use captures(address) instead of captures(none) for indirect args
While provenance cannot be captured through these arguments, the address / object identity can.
Fixes https://github.com/rust-lang/rust/issues/137668.
r? `@ghost`
cg_llvm: Assert that LLVM range-attribute values don't exceed 128 bits
The underlying implementation of `LLVMCreateConstantRangeAttribute` assumes that each of `LowerWords` and `UpperWords` points to enough u64 values to define an integer of the specified bit-length, and will encounter UB if that is not the case.
Our safe wrapper function always passes pointers to `[u64; 2]` arrays, regardless of the bit-length specified. That's fine in practice, because scalar primitives never exceed 128 bits, but it is technically a soundness hole in a safe function.
We can close the soundness hole by explicitly asserting `size_bits <= 128`. This is effectively just a stricter version of the existing check that the value must be small enough to fit in `c_uint`.
---
This is a narrower version of the fix in rust-lang/rust#145846.
The underlying implementation of `LLVMCreateConstantRangeAttribute` assumes
that each of `LowerWords` and `UpperWords` points to enough u64 values to
define an integer of the specified bit-length, and will encounter UB if that is
not the case.
Our safe wrapper function always passes pointers to `[u64; 2]` arrays,
regardless of the bit-length specified. That's fine in practice, because scalar
primitives never exceed 128 bits, but it is technically a soundness hole in a
safe function.
We can close the soundness hole by explicitly asserting `size_bits <= 128`.
This is effectively just a stricter version of the existing check that the
value must be small enough to fit in `c_uint`.
