Previously, it would only run on changes to subtrees, submodules, or select directories.
That made it so that changes to the compiler that broke tools would only be detected on a full bors merge.
This makes it so the tools builder runs by default, making it easier to catch breaking changes to clippy (which was the most effected).
See comment added for details on the test builder restriction. This is primarily
intended for macOS CI, but is likely to be a slight win on other builders too.
Re-enable submodule archive downloads.
This is effectively a revert of #98423 (though it keeps the `--depth 1` flag since that is still helpful).
GitHub has indicated that they have been working on the original issue, and my testing shows that the llvm-project archive download now succeeds 100% of the time.
This should save about a minute on every job.
The script wasn't referenced anywhere, and it's not useful anymore:
/opt/ghc is not present in new images, while /usr/share/dotnet is only
2.3 GB rather than 16 GB.
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
We currently upload approximately 166 GB/day into this bucket (estimate based on
duration of storage and total current size). My estimate is that this change
should decrease our costs (which are currently in credits) and is in the worst
case (if all objects are brought into hot storage due to unanticipated frequent
access) only going to add an additional ~$4 to the monthly bill. If access is
rare (as expected) to most objects then we expect to save approximately
~$350/month (after this change takes full effect in ~168 days).
Remove extra unshallow from cherry-pick checker
This is already done by 13db8440bb/src/ci/init_repo.sh (L32-L36) on the beta channel, and git throws an error if you attempt to unshallow an already non-shallow repository.
r? ```@pietroalbini```
This skips bumping Windows sccache because we run into compilation failures when
doing so (-m32 not supported by clang-cl?). Not clear on cause, but seems
easiest to just hold back.
This should avoid PGO-related failures encountered on Linux, and more broadly
seems like a good idea on other platforms as well (though it is likely not
necessary right this moment).
The only change here is a fix for `sys.platform` on Linux. Python 3.3
changed the API to return "linux" instead of "linux2"/"linux3", so this
commit uses `.startswith("python")` to make the code work on Python 3
without breaking Python 2.
