44 Commits

Author SHA1 Message Date
Nicholas Nethercote
c4bdd298fe Remove #[macro_use] extern crate tracing from rustc_incremental. 2024-05-23 18:02:17 +10:00
John Kåre Alsaker
6119763e19 Encode dep graph edges directly from the previous graph when promoting 2024-03-23 20:03:55 +01:00
Nicholas Nethercote
46f4983356 Adjust the has_errors* methods.
Currently `has_errors` excludes lint errors. This commit changes it to
include lint errors.

The motivation for this is that for most places it doesn't matter
whether lint errors are included or not. But there are multiple places
where they must be includes, and only one place where they must not be
included. So it makes sense for `has_errors` to do the thing that fits
the most situations, and the new `has_errors_excluding_lint_errors`
method in the one exceptional place.

The same change is made for `err_count`. Annoyingly, this requires the
introduction of `err_count_excluding_lint_errs` for one place, to
preserve existing error printing behaviour. But I still think the change
is worthwhile overall.
2024-02-22 08:03:47 +11:00
Nicholas Nethercote
1f9fa2305a Tweak error counting.
We have several methods indicating the presence of errors, lint errors,
and delayed bugs. I find it frustrating that it's very unclear which one
you should use in any particular spot. This commit attempts to instill a
basic principle of "use the least general one possible", because that
reflects reality in practice -- `has_errors` is the least general one
and has by far the most uses (esp. via `abort_if_errors`).

Specifics:
- Add some comments giving some usage guidelines.
- Prefer `has_errors` to comparing `err_count` to zero.
- Remove `has_errors_or_span_delayed_bugs` because it's a weird one: in
  the cases where we need to count delayed bugs, we should really be
  counting lint errors as well.
- Rename `is_compilation_going_to_fail` as
  `has_errors_or_lint_errors_or_span_delayed_bugs`, for consistency with
  `has_errors` and `has_errors_or_lint_errors`.
- Change a few other `has_errors_or_lint_errors` calls to `has_errors`,
  as per the "least general" principle.

This didn't turn out to be as neat as I hoped when I started, but I
think it's still an improvement.
2024-01-22 10:14:01 +11:00
Michael Woerister
739e5ef49e Split StableCompare trait out of StableOrd trait.
StableCompare is a companion trait to `StableOrd`. Some types like `Symbol` can be compared in a cross-session stable way, but their `Ord` implementation is not stable. In such cases, a `StableOrd` implementation can be provided to offer a lightweight way for stable sorting. (The more heavyweight option is to sort via `ToStableHashKey`, but then sorting needs to have access to a stable hashing context and `ToStableHashKey` can also be expensive as in the case of `Symbol` where it has to allocate a `String`.)
2024-01-04 13:32:42 +01:00
Nicholas Nethercote
8a9db25459 Remove more Session methods that duplicate DiagCtxt methods. 2023-12-24 08:17:47 +11:00
Nicholas Nethercote
99472c7049 Remove Session methods that duplicate DiagCtxt methods.
Also add some `dcx` methods to types that wrap `TyCtxt`, for easier
access.
2023-12-24 08:05:28 +11:00
Nicholas Nethercote
2c337a072c Rename HandlerInner::delayed_span_bugs as HandlerInner::span_delayed_bugs.
For reasons similar to the previous commit.
2023-12-02 09:01:34 +11:00
Ben Kimock
fbaa24ee35 Call FileEncoder::finish in rmeta encoding 2023-11-22 22:49:22 -05:00
Nicholas Nethercote
8da1b33153 Move a use to a more sensible spot.
I.e. in the source file where it's used.
2023-10-26 08:46:10 +11:00
John Kåre Alsaker
ba8d53dc8f Don't use a thread to load the dep graph 2023-09-24 00:30:43 +02:00
John Kåre Alsaker
2a5121b131 Avoid a source_span query when encoding Spans into query results 2023-09-08 02:08:52 +02:00
bjorn3
980143b50c Pass WorkProductMap to build_dep_graph instead of FxIndexMap
Constructing an FxIndexMap is useless work as the iteration order never
matters.
2023-08-13 16:07:48 +00:00
Andrew Xie
96b577860d Fixed failing test + minor cleanup 2023-06-04 21:55:32 -04:00
Andrew Xie
17412bae30 Removed use of iteration through a HashMap/HashSet in rustc_incremental and replaced with IndexMap/IndexSet 2023-06-04 21:54:28 -04:00
jyn
d5f2b8e5c6 Only depend on CFG_VERSION in rustc_interface
this avoids having to rebuild the whole compiler on each commit when
`omit-git-hash = false`.
2023-05-17 23:54:21 -05:00
John Kåre Alsaker
66d85438ca Remove QueryEngine trait 2023-04-26 07:46:13 +02:00
David Wood
2ff46641a9 incremental: migrate diagnostics
Migrate the `rustc_incremental` crate's diagnostics to translatable
diagnostic structs.

Signed-off-by: David Wood <david.wood@huawei.com>
2023-01-30 17:11:35 +00:00
Boxy
72d8879c29 make error_reported check for delayed bugs 2022-11-24 11:12:46 +00:00
Joshua Nelson
3c9765cff1 Rename debugging_opts to unstable_opts
This is no longer used only for debugging options (e.g. `-Zoutput-width`, `-Zallow-features`).
Rename it to be more clear.
2022-07-13 17:47:06 -05:00
David Wood
8371a036ea incr: cache dwarf objects in work products
Cache DWARF objects alongside object files in work products when those
exist so that DWARF object files are available for thorin in packed mode
in incremental scenarios.

Signed-off-by: David Wood <david.wood@huawei.com>
2022-07-06 11:15:13 +01:00
Nicholas Nethercote
bb02cc47c4 Move finish out of the Encoder trait.
This simplifies things, but requires making `CacheEncoder` non-generic.

(This was previously merged as commit 4 in #94732 and then was reverted
in #97905 because it caused a perf regression.)
2022-06-16 16:20:32 +10:00
Nicholas Nethercote
3186e311e5 Revert dc08bc51f2c58a0f5f815a07f9bb3d671153b5a1. 2022-06-10 11:58:29 +10:00
Nicholas Nethercote
7f51a1b976 Revert b983e42936feab29f6333e9835913afc6b4a394e. 2022-06-10 08:35:03 +10:00
Nicholas Nethercote
b983e42936 Rename rustc_serialize::opaque::Encoder as MemEncoder.
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).
2022-06-08 09:50:44 +10:00
Nicholas Nethercote
dc08bc51f2 Move finish out of the Encoder trait.
This simplifies things, but requires making `CacheEncoder` non-generic.
2022-06-08 09:21:05 +10:00
Nicholas Nethercote
1acbe7573d Use delayed error handling for Encodable and Encoder infallible.
There are two impls of the `Encoder` trait: `opaque::Encoder` and
`opaque::FileEncoder`. The former encodes into memory and is infallible, the
latter writes to file and is fallible.

Currently, standard `Result`/`?`/`unwrap` error handling is used, but this is a
bit verbose and has non-trivial cost, which is annoying given how rare failures
are (especially in the infallible `opaque::Encoder` case).

This commit changes how `Encoder` fallibility is handled. All the `emit_*`
methods are now infallible. `opaque::Encoder` requires no great changes for
this. `opaque::FileEncoder` now implements a delayed error handling strategy.
If a failure occurs, it records this via the `res` field, and all subsequent
encoding operations are skipped if `res` indicates an error has occurred. Once
encoding is complete, the new `finish` method is called, which returns a
`Result`. In other words, there is now a single `Result`-producing method
instead of many of them.

This has very little effect on how any file errors are reported if
`opaque::FileEncoder` has any failures.

Much of this commit is boring mechanical changes, removing `Result` return
values and `?` or `unwrap` from expressions. The more interesting parts are as
follows.
- serialize.rs: The `Encoder` trait gains an `Ok` associated type. The
  `into_inner` method is changed into `finish`, which returns
  `Result<Vec<u8>, !>`.
- opaque.rs: The `FileEncoder` adopts the delayed error handling
  strategy. Its `Ok` type is a `usize`, returning the number of bytes
  written, replacing previous uses of `FileEncoder::position`.
- Various methods that take an encoder now consume it, rather than being
  passed a mutable reference, e.g. `serialize_query_result_cache`.
2022-06-08 07:01:26 +10:00
bjorn3
e16c3b4a44 Make saved_file field of WorkProduct non-optional
A WorkProduct without a saved file is useless
2022-06-06 12:39:32 +00:00
Yuri Astrakhan
7e8201ae0a Spellchecking some comments
This PR attempts to clean up some minor spelling mistakes in comments
2022-03-30 01:39:38 -04:00
pierwill
41f76924d0 Document all public items in rustc_incremental
Also:

- Review and edit current docs
- Enforce documentation for crate

Co-authored-by: r00ster <r00ster91@protonmail.com>
Co-authored-by: Camille Gillot <gillot.camille@gmail.com>
2021-12-07 10:10:52 -06:00
Camille GILLOT
98007e2ce6 Drop the query result memmap before serializing it back. 2021-08-28 21:45:02 +02:00
Camille GILLOT
6b47e1ece8 Move save_in to file_format. 2021-08-28 21:45:02 +02:00
Camille GILLOT
3a6d5c2beb Avoid creating anonymous nodes with zero or one dependency. 2021-06-01 21:43:30 +02:00
Camille GILLOT
a50f1e949b Get rid of PreviousDepGraph. 2021-05-22 14:14:23 +02:00
Joshua Nelson
272015190d Add [TRACKED_NO_CRATE_HASH] and [SUBSTRUCT] directives
This is necessary for options that should invalidate the incremental
hash but *not* affect the crate hash (e.g. --remap-path-prefix).

This doesn't add `for_crate_hash` to the trait directly because it's not
relevant for *types*, only for *options*, which are fields on a larger
struct. Instead, it adds a new `SUBSTRUCT` directive for options, which
does take a `for_crate_hash` parameter.

- Use TRACKED_NO_CRATE_HASH for --remap-path-prefix
- Add test that `remap_path_prefix` is tracked
- Reduce duplication in the test suite to avoid future churn
2021-04-27 16:46:33 +00:00
Camille GILLOT
8ee9322c10 Also profile finishing the encoding. 2021-03-30 18:10:08 +02:00
Camille GILLOT
fe89f3236c Address review. 2021-03-30 18:10:08 +02:00
Camille GILLOT
6bfaf3a9cb Stream the dep-graph to a file. 2021-03-30 18:09:59 +02:00
Tyson Nottingham
09067db8a0 Serialize dependency graph directly from DepGraph
Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-12 22:20:29 -08:00
Tyson Nottingham
52f21791fb Serialize incr comp structures to file via fixed-size buffer
Reduce a large memory spike that happens during serialization by writing
the incr comp structures to file by way of a fixed-size buffer, rather
than an unbounded vector.

Effort was made to keep the instruction count close to that of the
previous implementation. However, buffered writing to a file inherently
has more overhead than writing to a vector, because each write may
result in a handleable error. To reduce this overhead, arrangements are
made so that each LEB128-encoded integer can be written to the buffer
with only one capacity and error check. Higher-level optimizations in
which entire composite structures can be written with one capacity and
error check are possible, but would require much more work.

The performance is mostly on par with the previous implementation, with
small to moderate instruction count regressions. The memory reduction is
significant, however, so it seems like a worth-while trade-off.
2021-01-11 12:13:22 -08:00
Dan Gohman
304643c00d Optimize away some fs::metadata calls.
This also eliminates a use of a `Path` convenience function, in support
of #80741, refactoring `std::path` to focus on pure data structures and
algorithms.
2021-01-06 08:31:25 -08:00
Joshua Nelson
622c48e4f1 Allow making RUSTC_BOOTSTRAP conditional on the crate name
The main change is that `UnstableOptions::from_environment` now requires
an (optional) crate name. If the crate name is unknown (`None`), then the new feature is not available and you still have to use `RUSTC_BOOTSTRAP=1`. In practice this means the feature is only available for `--crate-name`, not for `#![crate_name]`; I'm interested in supporting the second but I'm not sure how.

Other major changes:

- Added `Session::is_nightly_build()`, which uses the `crate_name` of
the session
- Added `nightly_options::match_is_nightly_build`, a convenience method
for looking up `--crate-name` from CLI arguments.
`Session::is_nightly_build()`should be preferred where possible, since
it will take into account `#![crate_name]` (I think).
- Added `unstable_features` to `rustdoc::RenderOptions`

  There is a user-facing change here: things like `RUSTC_BOOTSTRAP=0` no
  longer active nightly features. In practice this shouldn't be a big
  deal, since `RUSTC_BOOTSTRAP` is the opposite of stable and everyone
  uses `RUSTC_BOOTSTRAP=1` anyway.

- Add tests

  Check against `Cheat`, not whether nightly features are allowed.
  Nightly features are always allowed on the nightly channel.

- Only call `is_nightly_build()` once within a function

- Use booleans consistently for rustc_incremental

  Sessions can't be passed through threads, so `read_file` couldn't take a
  session. To be consistent, also take a boolean in `write_file_header`.
2020-11-07 13:45:11 -05:00
Dániel Buga
a21f3a76a9 Clean up encode_dep_graph 2020-10-28 17:22:05 +01:00
mark
9e5f7d5631 mv compiler to compiler/ 2020-08-30 18:45:07 +03:00