Use serde for target spec json deserialize
The previous manual parsing of `serde_json::Value` was a lot of complicated code and extremely error-prone. It was full of janky behavior like sometimes ignoring type errors, sometimes erroring for type errors, sometimes warning for type errors, and sometimes just ICEing for type errors (the icing on the top).
Additionally, many of the error messages about allowed values were out of date because they were in a completely different place than the FromStr impls. Overall, the system caused confusion for users.
I also found the old deserialization code annoying to read. Whenever a `key!` invocation was found, one had to first look for the right macro arm, and no go to definition could help.
This PR replaces all this manual parsing with a 2-step process involving serde.
First, the string is parsed into a `TargetSpecJson` struct. This struct is a 1:1 representation of the spec JSON. It already parses all the enums and is very simple to read and write.
Then, the fields from this struct are copied into the actual `Target`. The reason for this two-step process instead of just serializing into a `Target` is because of a few reasons
1. There are a few transformations performed between the two formats
2. The default logic is implemented this way. Otherwise all the default field values would have to be spelled out again, which is suboptimal. With this logic, they fall out naturally, because everything in the json struct is an `Option`.
Overall, the mapping is pretty simple, with the vast majority of fields just doing a 1:1 mapping that is captured by two macros. I have deliberately avoided making the macros generic to keep them simple.
All the `FromStr` impls now have the error message right inside them, which increases the chance of it being up to date. Some "`from_str`" impls were turned into proper `FromStr` impls to support this.
The new code is much less involved, delegating all the JSON parsing logic to serde, without any manual type matching.
This change introduces a few breaking changes for consumers. While it is possible to use this format on stable, it is very much subject to change, so breaking changes are expected. The hope is also that because of the way stricter behavior, breaking changes are easier to deal with, as they come with clearer error messages.
1. Invalid types now always error, everywhere. Previously, they would sometimes error, and sometimes just be ignored (which meant the users JSON was still broken, just silently!)
2. This now makes use of `deny_unknown_fields` instead of just warning on unused fields, which was done previously. Serde doesn't make it easy to get such warning behavior, which was the primary reason that this now changed. But I think error behavior is very reasonable too. If someone has random stale fields in their JSON, it is likely because these fields did something at some point but no longer do, and the user likely wants to be informed of this so they can figure out what to do.
This is also relevant for the future. If we remove a field but someone has it set, it probably makes sense for them to take a look whether they need this and should look for alternatives, or whether they can just delete it. Overall, the JSON is made more explicit.
This is the only expected breakage, but there could also be small breakage from small mistakes. All targets roundtrip though, so it can't be anything too major.
fixesrust-lang/rust#144153
The previous manual parsing of `serde_json::Value` was a lot of
complicated code and extremely error-prone. It was full of janky
behavior like sometimes ignoring type errors, sometimes erroring for
type errors, sometimes warning for type errors, and sometimes just
ICEing for type errors (the icing on the top).
Additionally, many of the error messages about allowed values were out
of date because they were in a completely different place than the
FromStr impls. Overall, the system caused confusion for users.
I also found the old deserialization code annoying to read. Whenever a
`key!` invocation was found, one had to first look for the right macro
arm, and no go to definition could help.
This PR replaces all this manual parsing with a 2-step process involving
serde.
First, the string is parsed into a `TargetSpecJson` struct. This struct
is a 1:1 representation of the spec JSON. It already parses all the
enums and is very simple to read and write.
Then, the fields from this struct are copied into the actual `Target`.
The reason for this two-step process instead of just serializing into a
`Target` is because of a few reasons
1. There are a few transformations performed between the two formats
2. The default logic is implemented this way. Otherwise all the default
field values would have to be spelled out again, which is
suboptimal. With this logic, they fall out naturally, because
everything in the json struct is an `Option`.
Overall, the mapping is pretty simple, with the vast majority of fields
just doing a 1:1 mapping that is captured by two macros. I have
deliberately avoided making the macros generic to keep them simple.
All the `FromStr` impls now have the error message right inside them,
which increases the chance of it being up to date. Some "`from_str`"
impls were turned into proper `FromStr` impls to support this.
The new code is much less involved, delegating all the JSON parsing
logic to serde, without any manual type matching.
This change introduces a few breaking changes for consumers. While it is
possible to use this format on stable, it is very much subject to
change, so breaking changes are expected. The hope is also that because
of the way stricter behavior, breaking changes are easier to deal with,
as they come with clearer error messages.
1. Invalid types now always error, everywhere. Previously, they would
sometimes error, and sometimes just be ignored (which meant the users
JSON was still broken, just silently!)
2. This now makes use of `deny_unknown_fields` instead of just warning
on unused fields, which was done previously. Serde doesn't make it
easy to get such warning behavior, which was the primary reason that
this now changed. But I think error behavior is very reasonable too.
If someone has random stale fields in their JSON, it is likely
because these fields did something at some point but no longer do,
and the user likely wants to be informed of this so they can figure
out what to do.
This is also relevant for the future. If we remove a field but
someone has it set, it probably makes sense for them to take a look
whether they need this and should look for alternatives, or whether
they can just delete it. Overall, the JSON is made more explicit.
This is the only expected breakage, but there could also be small
breakage from small mistakes. All targets roundtrip though, so it can't
be anything too major.
Ignore tests/run-make/link-eh-frame-terminator/rmake.rs when cross-compiling
The test tests/run-make/link-eh-frame-terminator/rmake.rs fails to link when cross-compiling. Therefore, it should be ignored in cross-compilation environments.
See [commit a27bdea](a27bdea4b7) and [commit 2beccc4](2beccc4d8e) for reference.
pass --gc-sections if -Zexport-executable-symbols is enabled and improve tests
Exported symbols are added as GC roots in linking, so `--gc-sections` won't hurt `-Zexport-executable-symbols`.
Fixes the run-make test to work on Linux. Enable the ui test on more targets.
cc rust-lang/rust#84161
Emit warning when there is no space between `-o` and arg
Closesrust-lang/rust#142812
`getopt` doesn't seem to have an API to check this, so we have to check the args manually.
r? compiler
Disable `tests/run-make/mte-ffi` because no CI runners have MTE extensions enabled
This PR disables the `tests/run-make/mte-ffi` run-make test because it is (1) broken, and (2) no CI runners have suitable MTE extensions enabled to run it correctly. This test being broken is tracked by https://github.com/rust-lang/rust/issues/141600.
The first commit also reverts `mte-ffi` changes introduced in rust-lang/rust#141576, as those fixes potentially changes the meaning of the test.
cc ```````@dheaton-arm``````` (as this test was introduced in https://github.com/rust-lang/rust/pull/128384)
### Context
In https://github.com/rust-lang/rust/pull/141576 when converting PR CI runners from x86_64 to aarch64 runners, it was noticed that this test failed on `aarch64-gnu-llvm-19-1` but not `aarch64-gnu`. It turns out that:
- `aarch64-gnu-llvm-19-1`
- Uses `gcc version 14.2.0 (Ubuntu 14.2.0-4ubuntu2)`
- Based on `lscpu` output, the hardware that was used for this runner does not have MTE enabled.
- `aarch64-gnu`
- Uses `gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)`
- Based on `lscpu` output, the hardware that was used for this runner does not have MTE enabled.
Based on [this comment](https://github.com/rust-lang/rust/pull/141576#issuecomment-2964179035), it seems like the test *requires* hardware with MTE extensions enabled to run properly (on ARMv8.5 or higher).
Furthermore, I believe this test does indeed have mismatched pointer type issues, i.e.
```
bar_string.c: In function ‘main’:
bar_string.c:36:9: error: assignment to ‘char *’ from incompatible pointer type ‘unsigned int *’ [-Wincompatible-pointer-types]
36 | ptr = (unsigned int *)((uintptr_t)ptr | 0x1fl << 56);
| ^
```
Which is only exposed by `aarch64-gnu-llvm-19-1` because `aarch64-gnu-llvm-19-1` uses **gcc 14.2.0** whereas `aarch64-gnu` uses **gcc 11.14.0**.
### Details
<details>
<summary>aarch64-gnu-llvm-19-1</summary>
```
gcc_version: Using built-in specs.
COLLECT_GCC=aarch64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-linux-gnu/14/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 14.2.0-4ubuntu2' --with-bugurl=file:///usr/share/doc/gcc-14/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --prefix=/usr --with-gcc-major-version-only --program-suffix=-14 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/libexec --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-libstdcxx-backtrace --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-offload-targets=nvptx-none=/build/gcc-14-T7YiXd/gcc-14-14.2.0/debian/tmp-nvptx/usr --enable-offload-defaulted --without-cuda-driver --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.2.0 (Ubuntu 14.2.0-4ubuntu2)
lscpu: Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: ARM
Model name: Neoverse-N2
Model: 0
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Stepping: r0p0
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16
L1d cache: 256 KiB (4 instances)
L1i cache: 256 KiB (4 instances)
L2 cache: 4 MiB (4 instances)
L3 cache: 128 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
```
</details>
<details>
<summary>aarch64-gnu</summary>
```
gcc_version: Using built-in specs.
COLLECT_GCC=aarch64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/11/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.4.0-1ubuntu1~22.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
lscpu: Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: ARM
Model: 0
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Stepping: r0p0
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16
L1d cache: 256 KiB (4 instances)
L1i cache: 256 KiB (4 instances)
L2 cache: 4 MiB (4 instances)
L3 cache: 128 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
```
</details>
### References
- https://www.kernel.org/doc/html/v5.10/arm64/memory-tagging-extension.html
---
cc ```````@marcoieni``````` as this PR reverts the `tests/run-make/mte-ffi` changes from rust-lang/rust#141576.
Adjust `run_make_support::symbols` helpers
Massage the `symbols` helpers to fill out {match all, match any} x {substring match, exact match}:
| | Substring match | Exact match |
|-----------|----------------------------------------|-------------------------------|
| Match any | `object_contains_any_symbol_substring` | `object_contains_any_symbol` |
| Match all | `object_contains_all_symbol_substring` | `object_contains_all_symbols` |
As I'd like to use `object_contains_all_symbols` for rust-lang/rust#143669.
As part of this:
- Rename `any_symbol_contains` to `object_contains_any_symbol_substring` for accuracy, as `any_symbol_contains` is actually "contains any matching substring".
- Remove `with_symbol_iter`.
Noticed while working on https://github.com/rust-lang/rust/pull/143669.
r? ``@ChrisDenton`` (or compiler)
Massage the `symbols` helpers to fill out {match all, match any} x
{substring match, exact match}:
| | Substring match | Exact match |
|-----------|----------------------------------------|-------------------------------|
| Match any | `object_contains_any_symbol_substring` | `object_contains_any_symbol` |
| Match all | `object_contains_all_symbol_substring` | `object_contains_all_symbols` |
As part of this, rename `any_symbol_contains` to
`object_contains_any_symbol_substring` for accuracy.
This stabilizes a subset of the `-Clink-self-contained` components on x64 linux:
the rust-lld opt-out.
The opt-in is not stabilized, as interactions with other stable flags require
more internal work, but are not needed for stabilizing using rust-lld by default.
Similarly, since we only switch to rust-lld on x64 linux, the opt-out is
only stabilized there. Other targets still require `-Zunstable-options`
to use it.
This stabilizes a subset of the `-Clinker-features` components on x64 linux:
the lld opt-out.
The opt-in is not stabilized, as interactions with other stable flags require
more internal work, but are not needed for stabilizing using rust-lld by default.
Similarly, since we only switch to rust-lld on x64 linux, the opt-out is
only stabilized there. Other targets still require `-Zunstable-options`
to use it.
See RUST-141600: this test is broken in two ways:
1. This test triggers `-Wincompatible-pointer-types` on GCC 14.
2. This test requires ARMv8.5+ w/ MTE extensions enabled, but GHA CI
runner hardware do not have this enabled.
Fix short linker error output
This PR does 2 things:
- It removes the braces when there's a single element. This is required since brace expansion (at least in bash and zsh) only triggers if there's at least 2 elements.
- It removes the extra `.rlib` suffixes of the elements. See https://github.com/rust-lang/rust/pull/135707#discussion_r2185212393 for context.
Running `cargo +stage1 build` on the following program:
```rust
unsafe extern "C" {
fn foo() -> libc::c_int;
}
fn main() {
let x = unsafe { foo() } as u32;
// println!("{}", data_encoding::BASE64.encode(&x.to_le_bytes()));
}
```
Gives the following diff before and after the PR:
```diff
-/tmp/foo/target/debug/deps/{liblibc-faf416f178830595.rlib}.rlib
+/tmp/foo/target/debug/deps/liblibc-faf416f178830595.rlib
```
Running on the same program with the additional dependency, we get the following diff:
```diff
-/tmp/foo/target/debug/deps/{liblibc-faf416f178830595.rlib,libdata_encoding-84bb5aadfa9e8839.rlib}.rlib
+/tmp/foo/target/debug/deps/{liblibc-faf416f178830595,libdata_encoding-84bb5aadfa9e8839}.rlib
```
New const traits syntax
This PR only affects the AST and doesn't actually change anything semantically.
All occurrences of `~const` outside of libcore have been replaced by `[const]`. Within libcore we have to wait for rustfmt to be bumped in the bootstrap compiler. This will happen "automatically" (when rustfmt is run) during the bootstrap bump, as rustfmt converts `~const` into `[const]`. After this we can remove the `~const` support from the parser
Caveat discovered during impl: there is no legacy bare trait object recovery for `[const] Trait` as that snippet in type position goes down the slice /array parsing code and will error
r? ``@fee1-dead``
cc ``@nikomatsakis`` ``@traviscross`` ``@compiler-errors``
Enable short-ice for Windows
Works fine for x64 without modifications.
x86 MSVC is still failing.
Addresses item in rust-lang/rust#128602
---
try-job: x86_64-mingw-*
try-job: x86_64-msvc-*
try-job: i686-msvc-*
Rollup of 9 pull requests
Successful merges:
- rust-lang/rust#142645 (Also emit suggestions for usages in the `non_upper_case_globals` lint)
- rust-lang/rust#142657 (mbe: Clean up code with non-optional `NonterminalKind`)
- rust-lang/rust#142799 (rustc_session: Add a structure for keeping both explicit and default sysroots)
- rust-lang/rust#142805 (Emit a single error when importing a path with `_`)
- rust-lang/rust#142882 (Lazy init diagnostics-only local_names in borrowck)
- rust-lang/rust#142883 (Add impl_trait_in_bindings tests from rust-lang/rust#61773)
- rust-lang/rust#142943 (Don't include current rustc version string in feature removed help)
- rust-lang/rust#142965 ([RTE-497] Ignore `c-link-to-rust-va-list-fn` test on SGX platform)
- rust-lang/rust#142972 (Add a missing mailmap entry)
r? `@ghost`
`@rustbot` modify labels: rollup
Enable reproducible-build-2 for Windows MSVC
Works with MSVC if instructing the linker to avoid timestamps and deleting the PDB between compilations.
Addresses item in rust-lang/rust#128602
---
try-job: x86_64-mingw-*
try-job: x86_64-msvc-*
try-job: i686-msvc-*
While cg_llvm is very lax about mismatched function signatures, cg_clif
will crash when there is any mismatch. It could be turned into an error,
but without Cranelift changes can't just be ignored.
Enable textrel-on-minimal-lib for Windows
`bin_name` needs to be used when building a runnable executable.
Addresses item in rust-lang/rust#128602
---
try-job: x86_64-mingw-*
try-job: x86_64-msvc-*
try-job: i686-msvc-*
Enable fmt-write-bloat for Windows
Seems to be working fine for MSVC once it has the correct binary name.
Addresses item in rust-lang/rust#128602
---
try-job: x86_64-mingw-*
try-job: x86_64-msvc-*
try-job: i686-msvc-*
