rust/codegen at 1ead4761e9e2f056385768614c23ffa7acb6a19e - rust

mirror of https://github.com/rust-lang/rust.git synced 2025-11-17 13:16:15 +00:00

History

bors 1ead4761e9 Auto merge of #119878 - scottmcm:inline-always-unwrap, r=workingjubilee

Tune the inlinability of `unwrap`

Fixes #115463
cc `@thomcc`

This tweaks `unwrap` on ~~`Option` &~~ `Result` to be two parts:
- `#[inline(always)]` for checking the discriminant
- `#[cold]` for actually panicking

The idea here is that checking the discriminant on a `Result` ~~or `Option`~~ should always be trivial enough to be worth inlining, even in `opt-level=z`, especially compared to passing it to a function.

As seen in the issue and codegen test, this will hopefully help particularly for things like `.try_into().unwrap()`s that are actually infallible, but in a way that's only visible with the inlining.

EDIT: I've restricted this to `Result` to avoid combining effects

2024-01-15 09:20:46 +00:00

auxiliary

…

avr

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

cffi

tests/codegen/c-variadic* -> cffi/c-variadic*

2023-07-29 18:34:41 -07:00

cross-crate-inlining

Add -Zcross-crate-inline-threshold=yes

2023-11-07 18:45:11 -05:00

debug-accessibility

Adapt debug-accessibility tests for msvc-style enums

2023-12-15 11:45:03 +00:00

dllimports

…

enum

tests/codegen/enum-* -> enum/enum-*

2023-07-29 18:34:41 -07:00

instrument-xray

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

intrinsics

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

issues

Add assume into NonZeroIntX::get

2024-01-06 14:26:37 +01:00

lib-optimizations

restrict test to x86-64

2023-09-02 13:42:58 +02:00

loongarch-abi

tests: Fix tests for LoongArch64

2023-08-18 14:56:53 +08:00

macos

Raise minimum supported macOS to 10.12

2023-09-23 19:14:25 -05:00

naked-fn

tests/codegen/naked-* -> naked-fn/naked-*

2023-07-29 18:34:41 -07:00

non-terminate

…

remap_path_prefix

…

repr

add support for rustc_abi(assert_eq) and use it to test some repr(transparent) cases

2023-09-07 09:14:29 +02:00

riscv-abi

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

sanitizer

CFI: Add char to CFI integer normalization

2023-12-07 11:28:16 -08:00

simd

Update std::simd usage and test outputs

2023-11-26 09:02:25 -05:00

simd-intrinsic

Fix alignment passed down to LLVM for simd_masked_load

2023-12-12 13:11:59 +01:00

src-hash-algorithm

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

unwind-abis

stabilize abi_thiscall

2023-08-07 14:11:03 +02:00

aarch64-struct-align-128.rs

aarch64-linux: properly handle 128bit aligned aggregates

2023-07-10 19:19:40 -04:00

abi-efiapi.rs

…

abi-main-signature-16bit-c-int.rs

Replace ignore-everything with only-arch

2023-07-29 15:49:07 -07:00

abi-main-signature-32bit-c-int.rs

llvm: Allow noundef in codegen tests

2024-01-02 18:02:17 +00:00

abi-repr-ext.rs

…

abi-sysv64.rs

…

abi-x86_64_sysv.rs

…

abi-x86-interrupt.rs

…

addr-of-mutate.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

adjustments.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

align-byval-vector.rs

implement vector-containing aggregate alignment for x86 darwin

2023-07-10 19:19:36 -04:00

align-byval.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

align-enum.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

align-fn.rs

…

align-offset.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

align-struct.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

alloc-optimisation.rs

…

array-clone.rs

…

array-codegen.rs

Copy 1-element arrays as scalars, not vectors

2023-10-07 00:10:32 -07:00

array-equality.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

array-map.rs

update array-map test for removed alloca

2023-07-10 19:19:35 -04:00

array-optimized.rs

Copy 1-element arrays as scalars, not vectors

2023-10-07 00:10:32 -07:00

ascii-char.rs

update test for new LLVM 18 codegen

2023-11-28 12:10:59 +00:00

asm-clobber_abi.rs

…

asm-clobbers.rs

…

asm-may_unwind.rs

…

asm-maybe-uninit.rs

Allow MaybeUninit in input and output of inline assembly

2023-08-23 21:57:18 +09:00

asm-multiple-options.rs

…

asm-options.rs

…

asm-powerpc-clobbers.rs

…

asm-sanitize-llvm.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

asm-target-clobbers.rs

…

async-fn-debug-awaitee-field.rs

s/generator/coroutine/

2023-10-20 21:14:01 +00:00

async-fn-debug-msvc.rs

s/generator/coroutine/

2023-10-20 21:14:01 +00:00

async-fn-debug.rs

…

atomic-operations.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

autovectorize-f32x4.rs

…

binary-search-index-no-bound-check.rs

…

bool-cmp.rs

…

box-uninit-bytes.rs

add regression test for not memcpying padding bytes

2023-08-22 15:10:56 +02:00

bpf-alu32.rs

…

branch-protection.rs

…

call-llvm-intrinsics.rs

…

call-metadata.rs

…

catch-unwind.rs

…

cdylib-external-inline-fns.rs

…

cf-protection.rs

…

cfguard-checks.rs

…

cfguard-disabled.rs

…

cfguard-nochecks.rs

…

cfguard-non-msvc.rs

…

char-ascii-branchless.rs

Refactor some char, u8 ascii functions to be branchless

2023-10-26 21:48:36 -04:00

codemodels.rs

…

coercions.rs

…

cold-call-declare-and-call.rs

Use preserve_mostcc for extern "rust-cold"

2023-08-26 17:42:59 -07:00

comparison-operators-2-tuple.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

comparison-operators-newtype.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

const_scalar_pair.rs

…

consts.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

coroutine-debug-msvc.rs

Rename lots of files that had generator in their name

2023-10-20 21:14:02 +00:00

coroutine-debug.rs

Rename lots of files that had generator in their name

2023-10-20 21:14:02 +00:00

dealloc-no-unwind.rs

…

debug-alignment.rs

…

debug-column-msvc.rs

…

debug-column.rs

…

debug-compile-unit-path.rs

…

debug-fndef-size.rs

Ignore MSVC in test.

2023-10-08 16:45:45 +00:00

debug-limited.rs

…

debug-line-directives-only.rs

…

debug-line-tables-only.rs

…

debug-linkage-name.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

debug-vtable.rs

…

debuginfo-constant-locals.rs

…

debuginfo-generic-closure-env-names.rs

…

debuginfo-inline-callsite-location.rs

Deduplicate inlined function debug info, but create a new lexical scope to child subsequent scopes and variables from colliding

2023-09-01 14:27:21 -07:00

deduced-param-attrs.rs

…

default-hidden-visibility.rs

Add unstable -Zdefault-hidden-visibility cmdline flag for rustc.

2023-12-13 21:14:23 +00:00

default-requires-uwtable.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

drop-in-place-noalias.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

drop.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

dst-vtable-align-nonzero.rs

…

dst-vtable-size-range.rs

…

ehcontguard_disabled.rs

Add support for generating the EHCont section

2023-11-21 13:41:23 -08:00

ehcontguard_enabled.rs

convert ehcont-guard to an unstable option

2023-11-21 14:24:23 -08:00

enable-lto-unit-splitting.rs

…

export-no-mangle.rs

…

external-no-mangle-fns.rs

…

external-no-mangle-statics.rs

…

fastcall-inreg.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

fatptr.rs

…

fewer-names.rs

Always name the return place.

2023-07-08 15:38:40 +02:00

float_math.rs

…

fn-impl-trait-self.rs

…

foo.s

…

force-frame-pointers.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

force-no-unwind-tables.rs

…

force-unwind-tables.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

frame-pointer.rs

…

function-arguments-noopt.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

function-arguments.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

function-return.rs

Add -Zfunction-return={keep,thunk-extern} option

2023-11-30 20:21:31 +01:00

gdb_debug_script_load.rs

…

generic-debug.rs

…

global_asm_include.rs

Replace ignore-everything with only-arch

2023-07-29 15:49:07 -07:00

global_asm_x2.rs

Replace ignore-everything with only-arch

2023-07-29 15:49:07 -07:00

global_asm.rs

Replace ignore-everything with only-arch

2023-07-29 15:49:07 -07:00

infallible-unwrap-in-opt-z.rs

Tune the inlinability of Result::unwrap

2024-01-12 10:57:58 -08:00

inherit_overflow.rs

Replace legacy ConstProp by GVN.

2023-12-24 20:08:57 +00:00

inline-always-works-always.rs

…

inline-debuginfo.rs

…

inline-function-args-debug-info.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

inline-hint.rs

…

instrument-coverage-off.rs

coverage: Add UI tests for values accepted by -Cinstrument-coverage

2023-10-23 17:41:40 +11:00

instrument-coverage.rs

coverage: Add UI tests for values accepted by -Cinstrument-coverage

2023-10-23 17:41:40 +11:00

instrument-mcount.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

integer-cmp.rs

…

integer-overflow.rs

…

internalize-closures.rs

…

intrinsic-no-unnamed-attr.rs

…

issue-97217.rs

Limit to LLVM 17.0.2 to work around WinEH codegen bug

2023-10-02 11:06:38 +02:00

iter-repeat-n-trivial-drop.rs

Get !nonnull metadata consistently in slice iterators, without needing assumes

2023-07-20 11:33:49 -07:00

layout-size-checks.rs

…

lifetime_start_end.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

link_section.rs

…

link-dead-code.rs

…

llvm_module_flags.rs

Add -Z llvm_module_flag

2023-11-11 19:48:47 -08:00

llvm-ident.rs

Support .comment section like GCC/Clang (!llvm.ident)

2023-07-21 22:01:50 +02:00

loads.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

local-generics-in-exe-internalized.rs

…

lto-removes-invokes.rs

…

mainsubprogram.rs

…

mainsubprogramstart.rs

…

match-optimized.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

match-optimizes-away.rs

…

match-unoptimized.rs

…

maybeuninit-rvo.rs

Add codegen test for RVO on MaybeUninit

2024-01-03 21:18:07 +01:00

mem-replace-big-type.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

mem-replace-simple-type.rs

Stop emitting non-power-of-two vectors in basic LLVM codegen

2023-08-25 20:06:57 -07:00

merge-functions.rs

…

method-declaration.rs

Add test for method debuginfo declaration.

2023-08-12 21:27:46 +08:00

mir_zst_stores.rs

…

mir-inlined-line-numbers.rs

…

move-before-nocapture-ref-arg.rs

add codegen test for #107436

2023-08-26 18:14:47 +09:00

move-operands.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

no_builtins-at-crate.rs

Add the no-builtins attribute to functions when no_builtins is applied at the crate level.

2023-07-18 22:15:47 +08:00

no-assumes-on-casts.rs

…

no-dllimport-w-cross-lang-lto.rs

…

no-jump-tables.rs

…

no-plt.rs

…

noalias-box-off.rs

…

noalias-box.rs

…

noalias-flag.rs

…

noalias-refcell.rs

…

noalias-rwlockreadguard.rs

…

noalias-unpin.rs

…

noreturn-uninhabited.rs

…

noreturnflag.rs

…

nounwind.rs

…

nrvo.rs

…

optimize-attr-1.rs

…

option-as-slice.rs

…

option-nonzero-eq.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

overaligned-constant.rs

tests: fix overaligned-constant to not over-specify getelementptr instr

2023-12-21 15:53:28 -05:00

packed.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

panic-abort-windows.rs

…

panic-in-drop-abort.rs

…

panic-unwind-default-uwtable.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

personality_lifetimes.rs

Make test compatible with 32-bit as well

2024-01-05 11:45:57 +01:00

pgo-counter-bias.rs

…

pgo-instrumentation.rs

…

pic-relocation-model.rs

…

pie-relocation-model.rs

…

ptr-arithmetic.rs

Tell LLVM that the negation in <*const T>::sub cannot overflow

2023-08-10 23:00:39 -07:00

ptr-read-metadata.rs

…

README.md

…

refs.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

repeat-trusted-len.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

scalar-pair-bool.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

set-discriminant-invalid.rs

…

slice_as_from_ptr_range.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

slice-as_chunks.rs

update tests that are ignored by debug

2023-09-01 04:01:54 +08:00

slice-indexing.rs

…

slice-init.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

slice-iter-fold.rs

Update the minimum external LLVM to 16.

2023-11-21 22:40:16 +01:00

slice-iter-len-eq-zero.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

slice-iter-nonnull.rs

Separate immediate and in-memory ScalarPair representation

2023-12-15 17:42:05 +01:00

slice-position-bounds-check.rs

…

slice-ref-equality.rs

Bless codegen tests.

2023-08-17 18:28:33 +00:00

slice-reverse.rs

…

slice-windows-no-bounds-check.rs

…

some-abis-do-extend-params-to-32-bits.rs

…

some-global-nonnull.rs

…

sparc-struct-abi.rs

llvm: Allow noundef in codegen tests

2024-01-02 18:02:17 +00:00

split-lto-unit.rs

…

sroa-fragment-debuginfo.rs

Restrict test to x86_64.

2023-08-26 22:55:52 +00:00

sse42-implies-crc32.rs

…

stack-probes-inline.rs

Enable stack probes on aarch64 for LLVM 18

2023-12-07 17:17:00 -08:00

stack-protector.rs

tests: be even more permissive on attributes in one test

2023-06-23 09:48:00 -04:00

static-relocation-model-msvc.rs

…

staticlib-external-inline-fns.rs

…

stores.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

swap-large-types.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

swap-small-types.rs

Stop emitting non-power-of-two vectors in basic LLVM codegen

2023-08-25 20:06:57 -07:00

target-cpu-on-functions.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

target-feature-inline-closure.rs

Don't accidentally detect the commit hash as an fadd instruction

2023-10-10 16:59:49 +11:00

target-feature-overrides.rs

…

thin-lto.rs

Add thinlto support to codegen, assembly and coverage tests

2023-11-30 18:48:03 +08:00

thread-local.rs

…

tied-features-strength.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

to_vec.rs

…

trailing_zeros.rs

add codegen test for issue 107554

2023-08-16 14:04:05 +09:00

transmute-optimized.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

transmute-scalar.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

try_identity.rs

…

try_question_mark_nop.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

tune-cpu-on-functions.rs

Automatically enable cross-crate inlining for small functions

2023-10-17 19:53:51 -04:00

tuple-layout-opt.rs

Remove some wasm/emscripten ignores

2023-08-25 19:48:20 -04:00

unchecked_shifts.rs

tests: update check for inferred nneg on zext

2023-11-13 10:43:33 -05:00

unchecked-float-casts.rs

…

uninit-consts.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

union-abi.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

unwind-and-panic-abort.rs

…

unwind-extern-exports.rs

…

unwind-extern-imports.rs

…

unwind-landingpad-cold.rs

Limit to LLVM 17.0.2 to work around WinEH codegen bug

2023-10-02 11:06:38 +02:00

unwind-landingpad-inline.rs

Limit to LLVM 17.0.2 to work around WinEH codegen bug

2023-10-02 11:06:38 +02:00

used_with_arg.rs

…

var-names.rs

Always name the return place.

2023-07-08 15:38:40 +02:00

vec_pop_push_noop.rs

Add invariant to Vec::pop that len < cap if pop successful

2023-10-16 18:49:25 +02:00

vec-as-ptr.rs

CHECK only for opaque ptr

2023-07-27 14:44:13 -07:00

vec-calloc.rs

Update the minimum external LLVM to 15

2023-07-27 14:07:08 -07:00

vec-in-place.rs

Update the minimum external LLVM to 16.

2023-11-21 22:40:16 +01:00

vec-iter-collect-len.rs

…

vec-iter.rs

mark vec::IntoIter pointers as !nonnull

2024-01-07 03:44:04 +01:00

vec-optimizes-away.rs

…

vec-reserve-extend.rs

Hint optimizer about reserved capacity

2023-11-02 00:52:06 +00:00

vec-shrink-panik.rs

use indexed loop instead of ptr bumping

2023-06-14 22:22:41 +02:00

vecdeque_no_panic.rs

…

vecdeque-nonempty-get-no-panic.rs

Add test to guard against VecDeque optimization regression

2023-09-21 20:42:21 -07:00

virtual-function-elimination-32bit.rs

…

virtual-function-elimination.rs

…

wasm_casts_trapping.rs

…

wasm_exceptions.rs

add codegen-test: wasm-exceptions

2023-07-01 20:24:15 +02:00

zip.rs

…

zst-offset.rs

Address review comments

2024-01-02 15:03:14 +01:00

README.md

The files here use the LLVM FileCheck framework, documented at https://llvm.org/docs/CommandGuide/FileCheck.html.

One extension worth noting is the use of revisions as custom prefixes for FileCheck. If your codegen test has different behavior based on the chosen target or different compiler flags that you want to exercise, you can use a revisions annotation, like so:

// revisions: aaa bbb
// [bbb] compile-flags: --flags-for-bbb

After specifying those variations, you can write different expected, or explicitly unexpected output by using <prefix>-SAME: and <prefix>-NOT:, like so:

// CHECK: expected code
// aaa-SAME: emitted-only-for-aaa
// aaa-NOT:                        emitted-only-for-bbb
// bbb-NOT:  emitted-only-for-aaa
// bbb-SAME:                       emitted-only-for-bbb