The implementation of the linkage attribute inside extern blocks defines
symbols starting with _rust_extern_with_linkage_. If someone tries to
also define this symbol you will get a symbol conflict or even an ICE.
By adding an unpredictable component to the symbol name, this becomes
less of an issue.
Make some "safe" llvm ops actually sound
Noticed while doing other refactorings
it may cause some extra unnecessary allocations, but the current use sites are rare ones anyway
#[used] currently is an alias for #[used(linker)] on all platforms
except ELF based ones where it is an alias for #[used(compiler)]. The
latter has surprising behavior and the LLVM LangRef explicitly states
that it "should only be used in rare circumstances, and should not be
exposed to source languages."
The reason #[used] still was an alias to #[used(compiler)] on ELF is
because the gold linker has issues with it. Luckily gold has been
deprecated with GCC 15 and seems to be unable to bootstrap rustc anyway.
As such we shouldn't really care about supporting gold.
- Rename `USED` to `USED_COMPILER` to better reflect its behavior.
- Reorder some items to group the used and allocator flags together
- Renumber them without gaps
Unfortunately, multiple people are reporting linker warnings related to
`__rust_no_alloc_shim_is_unstable` after this change. The solution isn't
quite clear yet, let's revert to green for now, and try a reland with a
determined solution for `__rust_no_alloc_shim_is_unstable`.
This reverts commit c8b7f32434c0306db5c1b974ee43443746098a92, reversing
changes made to 667247db71ea18c4130dd018d060e7f09d589490.
Autodiff batching
Enzyme supports batching, which is especially known from the ML side when training neural networks.
There we would normally have a training loop, where in each iteration we would pass in some data (e.g. an image), and a target vector. Based on how close we are with our prediction we compute our loss, and then use backpropagation to compute the gradients and update our weights.
That's quite inefficient, so what you normally do is passing in a batch of 8/16/.. images and targets, and compute the gradients for those all at once, allowing better optimizations.
Enzyme supports batching in two ways, the first one (which I implemented here) just accepts a Batch size,
and then each Dual/Duplicated argument has not one, but N shadow arguments. So instead of
```rs
for i in 0..100 {
df(x[i], y[i], 1234);
}
```
You can now do
```rs
for i in 0..100.step_by(4) {
df(x[i+0],x[i+1],x[i+2],x[i+3], y[i+0], y[i+1], y[i+2], y[i+3], 1234);
}
```
which will give the same results, but allows better compiler optimizations. See the testcase for details.
There is a second variant, where we can mark certain arguments and instead of having to pass in N shadow arguments, Enzyme assumes that the argument is N times longer. I.e. instead of accepting 4 slices with 12 floats each, we would accept one slice with 48 floats. I'll implement this over the next days.
I will also add more tests for both modes.
For any one preferring some more interactive explanation, here's a video of Tim's llvm dev talk, where he presents his work. https://www.youtube.com/watch?v=edvaLAL5RqU
I'll also add some other docs to the dev guide and user docs in another PR.
r? ghost
Tracking:
- https://github.com/rust-lang/rust/issues/124509
- https://github.com/rust-lang/rust/issues/135283
Avoid wrapping constant allocations in packed structs when not necessary
This way LLVM will set the string merging flag if the alloc is a nul terminated string, reducing binary sizes.
try-job: armhf-gnu
Currently, when declaring an extern weak function in Rust, we use the
following syntax:
```rust
unsafe extern "C" {
#[linkage = "extern_weak"]
static FOO: Option<unsafe extern "C" fn() -> ()>;
}
```
This allows runtime-checking the extern weak symbol through the Option.
When emitting LLVM-IR, the Rust compiler currently emits this static
as an i8, and a pointer that is initialized with the value of the global
i8 and represents the nullabilty e.g.
```
@FOO = extern_weak global i8
@_rust_extern_with_linkage_FOO = internal global ptr @FOO
```
This approach does not work well with CFI, where we need to attach CFI
metadata to a concrete function declaration, which was pointed out in
https://github.com/rust-lang/rust/issues/115199.
This change switches to emitting a proper function declaration instead
of a global i8. This allows CFI to work for extern_weak functions.
We keep initializing the Rust internal symbol with the function
declaration, which preserves the correct behavior for runtime checking
the Option.
Co-authored-by: Jakob Koschel <jakobkoschel@google.com>
Apply dllimport in ThinLTO
This partially reverts https://github.com/rust-lang/rust/pull/103353 by properly applying `dllimport` if `-Z dylib-lto` is passed. That PR should probably fully be reverted as it looks quite sketchy. We don't know locally if the entire crate graph would be statically linked.
This should hopefully be sufficient to make ThinLTO work for rustc on Windows.
r? ``@wesleywiser``
---
Edit: This PR is changed to just generally revert https://github.com/rust-lang/rust/pull/103353.
Explain why we retroactively change a static initializer to have a different type
I keep getting confused about it and in turn confused `@GuillaumeGomez` while trying to explain it badly
Pointers for variables all need to be in the same address space for
correct compilation. Therefore ensure that even if a global variable is
created in a different address space, it is casted to the default
address space before its value is used.
This is necessary for the amdgpu target and others where the default
address space for global variables is not 0.
For example `core` does not compile in debug mode when not casting the
address space to the default one because it tries to emit the following
(simplified) LLVM IR, containing a type mismatch:
```llvm
@alloc_0 = addrspace(1) constant <{ [6 x i8] }> <{ [6 x i8] c"bit.rs" }>, align 1
@alloc_1 = addrspace(1) constant <{ ptr }> <{ ptr addrspace(1) @alloc_0 }>, align 8
; ^ here a struct containing a `ptr` is needed, but it is created using a `ptr addrspace(1)`
```
For this to compile, we need to insert a constant `addrspacecast` before
we use a global variable:
```llvm
@alloc_0 = addrspace(1) constant <{ [6 x i8] }> <{ [6 x i8] c"bit.rs" }>, align 1
@alloc_1 = addrspace(1) constant <{ ptr }> <{ ptr addrspacecast (ptr addrspace(1) @alloc_0 to ptr) }>, align 8
```
As vtables are global variables as well, they are also created with an
`addrspacecast`. In the SSA backend, after a vtable global is created,
metadata is added to it. To add metadata, we need the non-casted global
variable. Therefore we strip away an addrspacecast if there is one, to
get the underlying global.
Allow disabling ASan instrumentation for globals
AddressSanitizer adds instrumentation to global variables unless the [`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes) attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to set this; previously it only had the desired effect on functions.
(cc https://github.com/rust-lang/rust/issues/39699)
the behavior of the type system not only depends on the current
assumptions, but also the currentnphase of the compiler. This is
mostly necessary as we need to decide whether and how to reveal
opaque types. We track this via the `TypingMode`.
Set "symbol name" in raw-dylib import libraries to the decorated name
`windows-rs` received a bug report that mixing raw-dylib generated and the Windows SDK import libraries was causing linker failures: <https://github.com/microsoft/windows-rs/issues/3285>
The root cause turned out to be #124958, that is we are not including the decorated name in the import library and so the import name type is also not being correctly set.
This change modifies the generation of import libraries to set the "symbol name" to the fully decorated name and correctly marks the import as being data vs function.
Note that this also required some changes to how the symbol is named within Rust: for MSVC we now need to use the decorated name but for MinGW we still need to use partially decorated (or undecorated) name.
Fixes#124958
Passing i686 MSVC and MinGW build: <https://github.com/rust-lang/rust/actions/runs/11000433888?pr=130586>
r? `@ChrisDenton`
AddressSanitizer adds instrumentation to global variables unless the
[`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes)
attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to
set this; previously it only had the desired effect on functions.
Remove macOS 10.10 dynamic linker bug workaround
Rust's current minimum macOS version is 10.12, so the hack can be removed. This PR also updates the `remove_dir_all` docs to reflect that all supported macOS versions are protected against TOCTOU race conditions (the fallback implementation was already removed in #127683).
try-job: dist-x86_64-apple
try-job: dist-aarch64-apple
try-job: dist-apple-various
try-job: aarch64-apple
try-job: x86_64-apple-1
Supertraits of `BuilderMethods` are all called `XyzBuilderMethods`.
Supertraits of `CodegenMethods` are all called `XyzMethods`. This commit
changes the latter to `XyzCodegenMethods`, for consistency.
