60 Commits

Author SHA1 Message Date
Camille GILLOT
fedd7785fe Access StableHashingContext in rustc_query_system. 2021-10-03 16:08:55 +02:00
bors
d6cd2c6c87 Auto merge of #82183 - michaelwoerister:lazier-defpathhash-loading2, r=wesleywiser
Simplify lazy DefPathHash decoding by using an on-disk hash table.

This PR simplifies the logic around mapping `DefPathHash` values encountered during incremental compilation to valid `DefId`s in the current session. It is able to do so by using an on-disk hash table encoding that allows for looking up values directly, i.e. without deserializing the entire table.

The main simplification comes from not having to keep track of `DefPathHashes` being used during the compilation session.
2021-09-18 14:37:39 +00:00
Tomasz Miąsko
b6b19f3b6c Use explicit log level in tracing instrument macro
Specify a log level in tracing instrument macro explicitly.

Additionally reduce the used log level from a default info level to a
debug level (all of those appear to be developer oriented logs, so there
should be no need to include them in release builds).
2021-09-15 19:02:10 +02:00
Michael Woerister
5445715c20 Remove RawDefId tracking infrastructure from incr. comp. framework.
This infrastructure is obsolete now with the new encoding scheme for
the DefPathHash->DefIndex maps in crate metadata.
2021-09-14 13:56:33 +02:00
Camille GILLOT
eeb3c8f4b7 Unify with_task functions.
Remove with_eval_always_task.
2021-08-22 20:23:32 +02:00
Camille GILLOT
45d6decc19 Remove try_mark_green_and_read. 2021-08-22 20:23:29 +02:00
Camille GILLOT
c3bf3969d4 Move assertion inwards.
`with_taks_impl` is only called from `with_eval_always_task` and
`with_task` . The former is only used in query invocation, while the
latter is also used to start the `tcx` and to trigger codegen.

This move should not change significantly the number of calls to this
assertion.
2021-08-22 20:23:29 +02:00
Aaron Hill
e6a5231238
Create QuerySideEffects and use it for diagnostics 2021-07-25 20:27:58 -05:00
Ryan Levick
b5bec17184 Add docs to new methods 2021-07-07 11:14:14 +02:00
Ryan Levick
6e33dce9c2 Profile incremental hashing 2021-07-07 10:43:30 +02:00
bors
e4a6032706 Auto merge of #85903 - bjorn3:rustc_serialize_cleanup, r=varkor
Remove unused functions and arguments from rustc_serialize
2021-06-07 14:40:26 +00:00
Camille GILLOT
b51f24f021 Make the reasoning more explicit. 2021-06-01 21:46:30 +02:00
Camille GILLOT
3a6d5c2beb Avoid creating anonymous nodes with zero or one dependency. 2021-06-01 21:43:30 +02:00
bjorn3
a2c4affe86 Remove unused functions and arguments from rustc_serialize 2021-06-01 19:29:11 +02:00
bors
f60a670256 Auto merge of #85319 - cjgillot:query-simp, r=Mark-Simulacrum
Simplification of query forcing

Extracted from #78780
2021-05-30 10:11:23 +00:00
Camille GILLOT
a50f1e949b Get rid of PreviousDepGraph. 2021-05-22 14:14:23 +02:00
Camille GILLOT
eb82187b13 Make the fast path faster. 2021-05-15 10:36:37 +02:00
Camille GILLOT
91444af87a Refactor try_mark_previous_green. 2021-05-15 10:27:27 +02:00
pierwill
0019ca9141 Fix outdated crate names in compiler docs
Changes `librustc_X` to `rustc_X`, only in documentation comments.
Plain code comments are left unchanged.

Also fix incorrect file paths.
2021-04-08 11:12:14 -05:00
Camille GILLOT
f3dde45d2a Enable debugging the dep-graph without debug-assertions.
It may also be useful in these cases,
and some CI configurations test without debug assertions.
2021-03-31 17:12:06 +02:00
Camille GILLOT
8ee9322c10 Also profile finishing the encoding. 2021-03-30 18:10:08 +02:00
Camille GILLOT
df24315ddf Adjust profiling. 2021-03-30 18:10:08 +02:00
Camille GILLOT
fe89f3236c Address review. 2021-03-30 18:10:08 +02:00
Camille GILLOT
65a8681a17 Add documentation. 2021-03-30 18:10:07 +02:00
Camille GILLOT
c5c935af92 Simplify tracking the encoder state. 2021-03-30 18:10:07 +02:00
Camille GILLOT
e1c99e5fcc Remove the parallel version. 2021-03-30 18:10:06 +02:00
Camille GILLOT
8208872fa2 Fix parallel compiler. 2021-03-30 18:10:06 +02:00
Camille GILLOT
cfe786e5e0 Fix tests.
Avoid invoking queries inside `check_paths`, since we are holding a lock
to the reconstructed graph.
2021-03-30 18:10:06 +02:00
Camille GILLOT
39b306a53d Do not allocate in decoder. 2021-03-30 18:10:05 +02:00
Camille GILLOT
6bfaf3a9cb Stream the dep-graph to a file. 2021-03-30 18:09:59 +02:00
Joshua Nelson
441dc3640a Remove (lots of) dead code
Found with https://github.com/est31/warnalyzer.

Dubious changes:
- Is anyone else using rustc_apfloat? I feel weird completely deleting
  x87 support.
- Maybe some of the dead code in rustc_data_structures, in case someone
  wants to use it in the future?
- Don't change rustc_serialize

  I plan to scrap most of the json module in the near future (see
  https://github.com/rust-lang/compiler-team/issues/418) and fixing the
  tests needed more work than I expected.

TODO: check if any of the comments on the deleted code should be kept.
2021-03-27 22:16:33 -04:00
Camille GILLOT
f96e960ccf Access the session directly from DepContext. 2021-02-20 22:53:46 +01:00
Camille GILLOT
b27266fdb2 Use a QueryContext for try_mark_green. 2021-02-19 17:51:56 +01:00
Camille GILLOT
ea3d465c95 Move try_load_from_on_disk_cache to the QueryContext. 2021-02-19 17:51:55 +01:00
Camille GILLOT
49c1b07a9e Decouple QueryContext from DepContext. 2021-02-19 17:51:49 +01:00
Camille GILLOT
211b05aef3 Don't require a QueryContext to access the DepGraph. 2021-02-19 17:51:49 +01:00
Eduard-Mihai Burtescu
6165d1cc72 Print -Ztime-passes (and misc stats/logs) on stderr, not stdout. 2021-02-18 14:13:38 +02:00
bors
c5a96fb797 Auto merge of #80957 - tgnottingham:direct_serialize_depgraph, r=michaelwoerister
Serialize dependency graph directly from DepGraph

Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-19 19:36:41 +00:00
LingMan
a56bffb4f9 Use Option::map_or instead of .map(..).unwrap_or(..) 2021-01-14 19:23:59 +01:00
Tyson Nottingham
09067db8a0 Serialize dependency graph directly from DepGraph
Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-12 22:20:29 -08:00
Camille GILLOT
016ea6b319 Use a side-table of consts instead of matching on the DepKind enum. 2021-01-08 17:48:02 +01:00
Camille GILLOT
d1220fdedf Simplify DepNodeParams. 2021-01-08 17:29:49 +01:00
Tyson Nottingham
03eb75f759 rustc_query_system: avoid race condition when using edge_count 2020-12-22 14:12:57 -08:00
Tyson Nottingham
22ed75158b rustc_query_system: add more comments for dependency graph 2020-12-22 14:12:57 -08:00
Tyson Nottingham
d6b2aaed7d rustc_query_system: rename intern_node to intern_new_node 2020-12-22 14:12:57 -08:00
Tyson Nottingham
712fcae13a rustc_query_system: remove inline annotation from edge_count
This isn't called frequently enough to justify inlining.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
4f76266295 rustc_query_system: minor cleanup
Remove effectively unused parameter and delete out of date comment.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
dd1ab840d2 rustc_query_system: use more space-efficient edges representation
Use single vector of edges rather than per-node vector. There is a small
hit to instruction counts (< 0.5%), but the memory savings make up for it.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
ea47269f5f rustc_query_system: share previous graph edges with current graph
Reduce memory consumption by sharing the previous dependency graph's
edges with the current graph when it is known to be valid to do so. It
is known to be valid whenever we mark a node green because all of its
dependencies were green. It is *not* known to be valid when we mark a
node green because we re-executed its query and its result was the same
as in the previous compilation session. In that case, the dependency set
might have changed (we don't try to determine whether or not it changed
and whether or not we can share).
2020-12-22 14:12:57 -08:00
Tyson Nottingham
f6d6b0c96d rustc_query_system: share previous graph data with current graph
Reduce memory consumption by taking advantage of red/green algorithm
properties to share the previous dependency graph's node data with the
current graph instead of storing node data redundantly. Red nodes can
share the `DepNode`, and green nodes can share the `DepNode` and
`Fingerprint`. Edges will be shared when possible in a later change.
2020-12-22 14:12:57 -08:00