Currently the default release profile enables LTO and single CGU builds,
which is very slow to build. Most tests are better run with
optimizations enabled since it allows testing a much larger number of
inputs, so it is inconvenient that building can sometimes take
significantly longer than the tests.
Remedy this by doing the following:
* Move the existing `release` profile to `release-opt`.
* With the above, the default `release` profile is untouched (16 CGUs
and thin local LTO).
* `release-checked` inherits `release`, so no LTO or single CGU.
This means that the simple `cargo test --release` becomes much faster
for local development. We are able to enable the other profiles as
needed in CI.
Tests should ideally still be run with `--profile release-checked` to
ensure there are no debug assetions or unexpected wrapping math hit.
`no-panic` still needs a single CGU, so must be run with `--profile
release-opt`. Since it is not possible to detect CGU or profilel
configuration from within build scripts, the `ENSURE_NO_PANIC`
environment variable must now always be set.
Introduce a way to ignore the results of icount regression tests, by
specifying `allow-regressions` in the pull request body. This should
apply to both pull requests and the merges based on them, since `gh pr
view` automatically handles both.
These benchmarks are fast to run, so the time cost here is pretty
minimal. Running softfloat benchmarks just ensures that we don't e.g.
test the performance of `_mm_sqrt_ss` rather than our implementation,
and running without softfloat gives us a way to see the effect of arch
intrinsics.
`--limit=1` seems to apply before `jq` filtering, meaning our
`WORKFLOW_NAME` ("CI") workflow may not appear in the input to the jq
query. Removing `--limit` provides a default amount of inputs that jq
can then filter from, so this works better.
Add support in `ci-util.py` for finding the most recent baseline and
downloading it, which new tests can then be compared against.
Arbitrarily select nightly-2025-01-16 as the rustc version to pin to in
benchmarks.
The test suite for this repo has quite a lot of tests, and it is
difficult to tell which contribute the most to the long CI runtime.
libtest does have an unstable flag to report test times, but that is
inconvenient to use because it needs to be passed only to libtest
binaries.
Switch to cargo-nextest [1] which provides time reporting and, overall,
a better test UI. It may also improve test runtime, though this seems
unlikely since we have larger test binaries with many small tests
(nextest benefits the most when there are larger binaries that can be
run in parallel).
For anyone running locally without, `run.sh` should still fall back to
`cargo test` if `cargo-nextest` is not available.
This diff includes some cleanup and consistency changes to other
CI-related files.
[1]: https://nexte.st
These crates take time building in CI, especially with the release
profile having LTO enabled, but there isn't really any reason to test
them with different features or in release mode. Disable this to save
some CI runtime.
Most users who are developing this crate are likely running on a Unix
system, since there isn't much to test against otherwise. For
convenience, enable the features required to run these tests by default.
Currently the features that control what we test against are
`build-musl` and `test-multiprecision`. I didn't name them very
consistently and there isn't really any reason for that.
Rename `test-multiprecision` to `build-mpfr` to better reflect what it
actually does and to be more consistent with `build-musl`.
A failing debug assertion or overflow without correctly wrapping or
saturating is a bug, but the `debug` profile that has these enabled does
not run enough test cases to hit edge cases that may trigger these. Add
a new `release-checked` profile that enables debug assertions and
overflow checks. This seems to only extend per-function test time by a
few seconds (or around a minute on longer extensive tests), so enable
this as the default on CI.
In order to ensure `no_panic` still gets checked, add a build-only step
to CI.
I do not believe Cargo separately caches crates with different sets of
features enabled. So, ensuring that tests run with `unstable-intrinsics`
are always grouped should slightly reduce runtime.
As an added benefit, all the debug mode tests run first so initial
feedback is available faster.
Add a CI job with a dynamically calculated matrix that runs extensive
jobs on changed files. This makes use of the new
`function-definitions.json` file to determine which changed files
require full tests for a routine to run.
We now have tests against our custom-built musl as well as tests against
MPFR. The tests against system musl covers less than those against
custom-built musl, and are less portable; there isn't much benefit to
keeping them around so just remove them.
Introduce a Cargo feature to enable or disable architecture-specific
features (SIMD, assembly), which is on by default. This allows for more
fine grained control compared to relying on the `force-soft-floats`
feature.
Similar to "unstable-intrinsics", introduce a build.rs config option for
`unstable-intrinsics AND NOT force-soft-floats`, which makes this easier
to work with in code.
Effectively, this allows moving our non-additive Cargo feature
(force-soft-floats) to a positive one by default, allowing for an
override when needed.
We currently have a non-additive feature, "force-soft-floats", and we
will need to gain another "no-f16-f128". This makes `cfg` usage in code
somewhat confusing and redundant.
Use `build.rs` to figure out if "unstable-intrinsics" is enabled while
"force-soft-floats" is not enabled and if so, emit a cfg
`intrinsics_enabled`. This is cleaner to use and should make adding more
features easier to reason about.
Also use this as an opportunity to eliminate the build.rs from the
compiler-builtins test crate, replaced with the `[lints]` table in
Cargo.toml.
Currently there is a single feature called "unstable" that is used to
control whether intrinsics may be called. In anticipation of adding
other unstable features that we will want to control separately, create
a new feature called "unstable-intrinsics" that is enabled by
"unstable". Then move everything gated by "unstable" to
"unstable-intrinsics".
CI for aarch64 Linux is significantly slower than the others. Adjust how
iteration selection is done to better handle this case, which also
simplifies things.
Also set the `EMULATED` environment variable in Docker to be more
accurate, and reindents run-docker.sh.
We will have more test features in the near future, and it would be nice
for them all to have a common `test-` prefix. Reverse the existing
feature so this is the case.
There isn't any reason for this feature to be exposed or part of the
build script. Move it to a separate crate.
We will also want more tests that require some support functions; this
will create a place for them.
