This also allows reusing the same generator logic between logspace tests
and extensive tests, so comes with a nice bit of cleanup.
Changes:
* Make the generator part of `CheckCtx` since a `Generator` and
`CheckCtx` are almost always passed together.
* Rename `domain_logspace` to `spaced` since this no longer only
operates within a domain and we may want to handle integer spacing.
* Domain is now calculated at runtime rather than using traits, which is
much easier to work with.
* With the above, domains for multidimensional functions are added.
* The extensive test generator code tests has been combined with the
domain_logspace generator code. With this, the domain tests have just
become a subset of extensive tests. These were renamed to "quickspace"
since, technically, the extensive tests are also "domain" or "domain
logspace" tests.
* Edge case generators now handle functions with multiple inputs.
* The test runners can be significantly cleaned up and deduplicated.
Introduce a simple binary that can run arbitrary input against any of
the available implementations (musl, MPFR, our libm). This provides an
easy way to check results, or run specific cases against a debugger.
Examples:
$ cargo run -p util -- eval libm pow 1.6 2.4
3.089498284311124
$ cargo run -p util -- eval mpfr pow 1.6 2.4
3.089498284311124
$ cargo run -p util -- eval musl tgamma 1.2344597839132
0.9097442657960874
$ cargo run -p util -- eval mpfr tgamma 1.2344597839132
0.9097442657960874
$ cargo run -p util -- eval libm tgamma 1.2344597839132
0.9097442657960871
$ cargo run -p util -- eval musl sincos 3.1415926535
(8.979318433952318e-11, -1.0)
Most users who are developing this crate are likely running on a Unix
system, since there isn't much to test against otherwise. For
convenience, enable the features required to run these tests by default.
Currently the features that control what we test against are
`build-musl` and `test-multiprecision`. I didn't name them very
consistently and there isn't really any reason for that.
Rename `test-multiprecision` to `build-mpfr` to better reflect what it
actually does and to be more consistent with `build-musl`.
There was a recent failure from the random tests:
---- mp_random_exp2f stdout ----
Random Mpfr exp2f arg 1/1: 10000 iterations (10000 total) using `LIBM_SEED=fqgMuzs6eqH1VZSEmQpLnThnaIyRUOWe`
thread 'mp_random_exp2f' panicked at crates/libm-test/tests/multiprecision.rs:41:49:
called `Result::unwrap()` on an `Err` value:
input: (127.97238,) (0x42fff1dc,)
expected: 3.3383009e38 0x7f7b2556
actual: inf 0x7f800000
Caused by:
mismatched infinities
Add an xfail for mismatched infinities on i586.
`rint` had a couple recent failures from the random tests:
---- mp_random_rint stdout ----
Random Mpfr rint arg 1/1: 10000 iterations (10000 total) using `LIBM_SEED=Fl1f69DaJnwkHN2FeuCXaBFRvJYsPvEY`
thread 'mp_random_rint' panicked at crates/libm-test/tests/multiprecision.rs:41:49:
called `Result::unwrap()` on an `Err` value:
input: (-849751480.5001163,) (0xc1c95316dc4003d0,)
expected: -849751481.0 0xc1c95316dc800000
actual: -849751480.0 0xc1c95316dc000000
Caused by:
ulp 8388608 > 100000
And:
---- mp_random_rint stdout ----
Random Mpfr rint arg 1/1: 10000 iterations (10000 total) using `LIBM_SEED=XN7VCGhX3Wu6Mzn8COvJPITyZlGP7gN7`
thread 'mp_random_rint' panicked at crates/libm-test/tests/multiprecision.rs:41:49:
called `Result::unwrap()` on an `Err` value:
input: (-12493089.499809155,) (0xc167d4242ffe6fc5,)
expected: -12493089.0 0xc167d42420000000
actual: -12493090.0 0xc167d42440000000
Caused by:
ulp 536870912 > 100000
It seems we just implement an incorrect rounding mode. Replace the
existing `rint` override with an xfail if the difference is 0.0 <= ε <=
1.0.
`compiler_builtins` exposes an `extern "C"` version of `libm` routines,
so add the same here. There really isn't much to test here (unless we
later add tests against C `libm` suites), but one nice benefit is this
gives us a library with unmangled names that is easy to `objdump`. In
accordance with that, also update `cb` to be a `staticlib`.
Unfortunately this also means we have to remove it from the workspace,
since Cargo doesn't allow setting `panic = "abort"` for a single crate.
`ExpInt` is likely to only have performance benefits on 16-bit
platforms, but makes working with the exponent more difficult. It seems
like a worthwhile tradeoff to instead just use `i32`, so do that here.
Update test traits to support `f16` and `f128`, as applicable. Add the
new routines (`fabs` and `copysign` for `f16` and `f128`) to the list of
all operations.
Add a generator that will test all inputs for input spaces `u32::MAX` or
smaller (e.g. single-argument `f32` routines). For anything larger,
still run approximately `u32::MAX` tests, but distribute inputs evenly
across the function domain.
Since we often only want to run one of these tests at a time, this
implementation parallelizes within each test using `rayon`. A custom
test runner is used so a progress bar is possible.
Specific tests must be enabled by setting the `LIBM_EXTENSIVE_TESTS`
environment variable, e.g.
LIBM_EXTENSIVE_TESTS=all_f16,cos,cosf cargo run ...
Testing on a recent machine, most tests take about two minutes or less.
The Bessel functions are quite slow and take closer to 10 minutes, and
FMA is increased to run for about the same.
New random seeds seem to indicate that this test does have some more
failures, this is a recent failure on i586:
---- musl_random_jnf stdout ----
Random Musl jnf arg 1/2: 100 iterations (10000 total) using `LIBM_SEED=nLfzQ3U1OBVvqWaMBcto84UTMsC5FIaC`
Random Musl jnf arg 2/2: 100 iterations (10000 total) using `LIBM_SEED=nLfzQ3U1OBVvqWaMBcto84UTMsC5FIaC`
thread 'musl_random_jnf' panicked at crates/libm-test/tests/compare_built_musl.rs:43:51:
called `Result::unwrap()` on an `Err` value:
input: (205, 5497.891) (0x000000cd, 0x45abcf21)
expected: 7.3291517e-6 0x36f5ecef
actual: 7.331668e-6 0x36f6028c
Caused by:
ulp 5533 > 4000
It seems unlikely that `jn` would somehow have better precision than
`j0`/`j1`, so just use the same precision.
The `support` module that this feature makes public will be useful for
implementations in `compiler-builtins`, not only for testing. Give this
feature a more accurate name.
Currently, all inputs are generated and then cached. This works
reasonably well but it isn't very configurable or extensible (adding
`f16` and `f128` is awkward).
Replace this with a trait for generating random sequences of tuples.
This also removes possible storage limitations of caching all inputs.
Introduce the `KnownSize` iterator wrapper, which allows providing the
size at construction time. This provides an `ExactSizeIterator`
implemenation so we can check a generator's value count during testing.
Currently, tests use a handful of constants to determine how many
iterations to perform: `NTESTS`, `AROUND`, and `MAX_CHECK_POINTS`. This
configuration is not very straightforward to adjust and needs to be
repeated everywhere it is used.
Replace this with new functions in the `run_cfg` module that determine
iteration counts in a more reusable and documented way.
This only updates `edge_cases` and `domain_logspace`, `random` is
refactored in a later commit.
Occasionally it is useful to see some information from running tests
without making everything noisy from `--nocapture`. Add a function to
log this kind of output to a file, and print the file as part of CI.
Currently our implementations for `abs` and `copysign` are defined on
the trait, and these are then called from `generic`. It would be better
to call core's `.abs()` / `.copysign(y)`, but we can't do this in the
generic because calling the standalone function could become recursive
(`fabsf` becomes `intrinsics::fabsf32`, that may lower to a call to
`fabsf`).
Change this so the traits uses the call to `core` if available, falling
back to a call to the standalone generic function.
In practice the recursion isn't likely to be a problem since LLVM
probably always lowers `abs`/`copysign` to assembly, but this pattern
should be more correct for functions that we will add in the future
(e.g. `fma`).
This should eventually be followed by a change to call the trait methods
rather than `fabs`/`copysign` directly.
Once we start addinf `f16` and `f128` routines, we will need to have
this cfg for almost all uses of `for_each_function`. Rather than needing
to specify this each time, always emit `#[cfg(f16_enabled)]` or
`#[cfg(f128_enabled)]` for each function that uses `f16` or `f128`,
respectively.
Now that we are using rustdoc output to locate public functions, the
test is indicating a few that were missed since they don't have their
own function. Update everything to now include the following routines:
* `erfc`
* `erfcf`
* `y0`
* `y0f`
* `y1`
* `y1f`
* `yn`
* `ynf`
Currently `logspace` does a lossy cast from `F::Int` to `usize`. This
could be problematic in the rare cases that this is called with a step
count exceeding what is representable in `usize`.
Resolve this by instead adding bounds so the float's integer type itself
can be iterated.
These types from `libm-macros` provide a way to get information about an
operation at runtime, rather than only being encoded in the type system.
Include the file and reexport relevant types.
This will enable us to `include!` the file to access these types in
`libm-test`, rather than somehow reproducing the types as part of the
macro. Ideally `libm-test` would just `use` the types from `libm-macros`
but proc macro crates cannot currently export anything else.
This also adjusts naming to closer match the scheme described in
`libm_test::op`.
