3176 Commits

Author SHA1 Message Date
Thomas de Zeeuw
224acd2500 Pin Future to stack in #[tokio::test]
Instead of boxing it.
2022-11-29 14:28:13 +00:00
Thomas de Zeeuw
2fcc6c2cb0 Box Futures in #[tokio::test]
This reduces the amount of copies of the Runtime::block_on and related
functions the compiler has to generate and LLVM process. We've seen it
reduce the compilation time of our tests (some 1900 of them) from 40s
down to 12s, with no impact on the runtime of tests.

Below is an output of llvm-lines for our tests.

Before:

  Lines                  Copies                Function name
  -----                  ------                -------------
  8954414                156577                (TOTAL)
   984626 (11.0%, 11.0%)   9289 (5.9%,  5.9%)  std:🧵:local::LocalKey<T>::try_with
   648093 (7.2%, 18.2%)    1857 (1.2%,  7.1%)  tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
   557100 (6.2%, 24.5%)    3714 (2.4%,  9.5%)  tokio::park:🧵:CachedParkThread::block_on
   551679 (6.2%, 30.6%)    7430 (4.7%, 14.2%)  tokio::coop::with_budget::{{closure}}
   514389 (5.7%, 36.4%)    3714 (2.4%, 16.6%)  tokio::runtime::scheduler::current_thread::Context::enter
   326832 (3.6%, 40.0%)    1857 (1.2%, 17.8%)  tokio::runtime::scheduler::current_thread::CurrentThread::block_on
   291549 (3.3%, 43.3%)    1857 (1.2%, 19.0%)  tokio::runtime::scheduler::current_thread::CoreGuard::enter
   261907 (2.9%, 46.2%)    7430 (4.7%, 23.7%)  tokio::coop::budget
   189468 (2.1%, 48.3%)    7430 (4.7%, 28.5%)  tokio::coop::with_budget
   137418 (1.5%, 49.8%)    3714 (2.4%, 30.8%)  tokio::runtime::enter::Enter::block_on
   126276 (1.4%, 51.3%)    1857 (1.2%, 32.0%)  tokio::runtime::Runtime::block_on
   124419 (1.4%, 52.6%)    1857 (1.2%, 33.2%)  tokio::macros::scoped_tls::ScopedKey<T>::set
   118897 (1.3%, 54.0%)    3715 (2.4%, 35.6%)  core::option::Option<T>::or_else
   111420 (1.2%, 55.2%)    1857 (1.2%, 36.8%)  tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
   109408 (1.2%, 56.4%)    2105 (1.3%, 38.1%)  <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   105893 (1.2%, 57.6%)    9289 (5.9%, 44.0%)  std:🧵:local::LocalKey<T>::with
    96564 (1.1%, 58.7%)    1857 (1.2%, 45.2%)  tokio::runtime::scheduler::current_thread::Context::run_task
    90993 (1.0%, 59.7%)    7428 (4.7%, 50.0%)  tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
    90515 (1.0%, 60.7%)    2105 (1.3%, 51.3%)  core::pin::Pin<&mut T>::map_unchecked_mut
    89136 (1.0%, 61.7%)    1857 (1.2%, 52.5%)  tokio::runtime::scheduler::multi_thread::MultiThread::block_on

After:

  Lines                  Copies               Function name
  -----                  ------               -------------
  3188618                41634                (TOTAL)
   109408 (3.4%,  3.4%)   2105 (5.1%,  5.1%)  <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
    90515 (2.8%,  6.3%)   2105 (5.1%, 10.1%)  core::pin::Pin<&mut T>::map_unchecked_mut
    56220 (1.8%,  8.0%)   1874 (4.5%, 14.6%)  alloc::boxed::Box<T>::pin
    48333 (1.5%,  9.5%)   2179 (5.2%, 19.8%)  core::ops::function::FnOnce::call_once
    28587 (0.9%, 10.4%)      1 (0.0%, 19.8%)  XXXXXXXXXXXXXXXXXXX
    18730 (0.6%, 11.0%)   1873 (4.5%, 24.3%)  alloc::boxed::Box<T,A>::into_pin
    16190 (0.5%, 11.5%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    15870 (0.5%, 12.0%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    15250 (0.5%, 12.5%)      1 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXX
    12801 (0.4%, 12.9%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12801 (0.4%, 13.3%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12630 (0.4%, 13.7%)   2105 (5.1%, 29.4%)  <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::{{closure}}
    12613 (0.4%, 14.1%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12613 (0.4%, 14.5%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12613 (0.4%, 14.9%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12613 (0.4%, 15.3%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    11395 (0.4%, 15.7%)     96 (0.2%, 29.7%)  alloc::alloc::box_free
    11364 (0.4%, 16.0%)   1891 (4.5%, 34.2%)  <T as core::convert::Into<U>>::into
    11238 (0.4%, 16.4%)   1873 (4.5%, 38.7%)  alloc::boxed::<impl core::convert::From<alloc::boxed::Box<T,A>> for core::pin::Pin<alloc::boxed::Box<T,A>>>::from
    10735 (0.3%, 16.7%)      2 (0.0%, 38.7%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Note that I have replaced our test functions with XXX. As you can
clearly see they're not in the top 20 in the before output, while
they're in the after oput.

Further note that the amount of copies have been reduced from 156577 to
41634.
2022-11-29 14:28:13 +00:00
Tymoteusz Wiśniewski
28ec4a6161
net: update try_io docs with interest limitations (#5222) 2022-11-28 10:38:40 +01:00
Loong Dai
939b5bb42f
tests: fix a typo (#5236)
Signed-off-by: Loong <loong.dai@intel.com>
2022-11-28 10:37:22 +01:00
Abutalib Aghayev
718d6ce8ca
task: fix the incomplete/wrong description of JOIN_WAKER bit. (#5217)
* task: fix the incomplete/wrong description of JOIN_WAKER bit (#5217)
2022-11-24 15:47:15 -05:00
Tymoteusz Wiśniewski
e316428210
net: replace socket with pipe in named pipe docs (#5221) 2022-11-23 13:48:30 +01:00
Taiki Endo
fc83e01949
chore: add tokio_no_atomic_u64 cfg (#5226) 2022-11-23 19:00:57 +09:00
Loong Dai
3a5f7b7f2f
examples: update hello world comment (#5219) 2022-11-23 11:00:25 +01:00
Loong Dai
6da81471f9
signal: fix a typo (#5224) 2022-11-23 10:59:44 +01:00
Kenny Kerr
299bd6aee3
net: replace winapi with windows-sys (#5204) 2022-11-22 16:23:26 +00:00
Loong Dai
e14307393a
runtime: fix typo in runtime builder docs (#5218)
Signed-off-by: Loong <loong.dai@intel.com>
2022-11-22 08:24:12 +00:00
Alice Ryhl
45e37dbfa2
runtime: reduce codegen per task (#5213)
This PR should hopefully reduce the amount of code generated per
future-type spawned on the runtime. The following methods are no longer generic:

* `try_set_join_waker`
* `remote_abort`
* `clone_waker`
* `drop_waker`
* `wake_by_ref`
* `wake_by_val`

A new method is added to the vtable called schedule, which is used when a task
should be scheduled on the runtime. E.g. wake_by_ref will call it if the state change
says that the task needs to be scheduled. However, this method is only generic over
the scheduler, and not the future type, so it also isn't generated for every task.

Additionally, one of the changes involved in the above makes it possible to remove
the id field from JoinHandle and AbortHandle.
2022-11-21 15:38:22 -08:00
Jiahao XU
304b5152a7
sync: add owned future for CancellationToken (#5153) 2022-11-21 15:03:09 +00:00
Joonas Koivunen
f15d14ee91
process: support ChildStdin::poll_write_vectored on unix (#5216) 2022-11-21 13:57:24 +00:00
Alice Ryhl
6a2cd9a652
doc: update parking_lot info in feature list (#5215) 2022-11-21 13:42:55 +01:00
Tymoteusz Wiśniewski
2682c505e8
net: fix named pipe connect (#5208) 2022-11-20 21:02:13 +00:00
Carl Lerche
808d52563e
ci: run tests on ARM and i686 using cross (#5196)
This patch updates CI to use `cross` to run Tokio tests on virtualized
ARM and i686 VMs. Because ipv6 doesn't work on Github action running in
a docker instance, those tests are disabled
2022-11-18 14:02:53 -08:00
Alice Ryhl
bf31759bff
chore: prepare Tokio v1.22.0 (#5203) tokio-1.22.0 2022-11-18 13:15:06 -08:00
John DiSanti
d65826236b
ci: remove libc types from external types allow list (#5197) 2022-11-17 11:57:55 +01:00
Lucas Kent
1cbbcc9ad5
sync: specify return type of oneshot::Receiver in docs (#5198) 2022-11-17 11:56:56 +01:00
Abutalib Aghayev
a668020150
net: remove libc type leakage in a public API (#5191) 2022-11-15 23:28:33 +01:00
Alice Ryhl
01f0193971
chore: fix compilation on master (#5190) 2022-11-13 13:21:08 +00:00
Abutalib Aghayev
71bd49e146
task: add task::id() and task::try_id() (#5171) 2022-11-13 14:18:42 +01:00
Artyom Kozhemiakin
582d512907
sync: add mpsc::WeakUnboundedSender (#5189)
Signed-off-by: Artyom Kozhemiakin <xenus.t@gmail.com>
2022-11-12 22:29:13 +00:00
Carl Lerche
b7812c85ca
rt: fix LocalSet drop in thread local (#5179)
`LocalSet` cleans up any tasks that have not yet been completed when it is
dropped. Previously, this cleanup process required access to a thread-local.
Suppose a `LocalSet` is stored in a thread-local itself. In that case, when it is
dropped, there is no guarantee the drop implementation will be able to
access the internal `LocalSet` thread-local as it may already have been
destroyed.

The internal `LocalSet` thread local is mainly used to avoid writing unsafe
code. All `LocalState` that cannot be moved across threads is stored in the
thread-local and accessed on demand.

This patch moves this local-only state into the `LocalSet`'s "shared" struct.
Because this struct *is* `Send`, the local-only state is stored in `UnsafeCell`,
and callers must ensure not to touch it from other threads.

A debug assertion is added to enforce this requirement in tests.

Fixes #5162
2022-11-10 10:06:25 -08:00
Abutalib Aghayev
9e3fb1673a
rt: move CoreStage methods to Core (#5182) 2022-11-10 13:15:30 +00:00
Carl Lerche
53cba023da
rt: fix accidental unsetting of current handle (#5178)
An earlier change updated `enter_runtime` to also set the current
handle. However, the change did not store the `SetCurrentGuard`, so the
"current handle" was immediately unset. This patch stores the
`SetCurrentGuard` in the `EnterRuntimeGuard`.

No existing test exposed this bug because all tests went via `Runtime`
instead of `Handle`. Currently, `Runtime` is still explicitly setting
the handle before entering runtime, so all tests still passed. A new
test is added that covers the case of calling `Handle::block_on` and
accessing the current handle.
2022-11-09 12:13:30 -08:00
Carl Lerche
236d026667
rt: combine context and entered thread-locals (#5168)
A previous patch moved code related to entering a runtime into the
context module but did not change anything. This patch combines both
thread-local variables.
2022-11-07 13:35:54 -08:00
Alice Ryhl
909439c9f5
task: elaborate safety comments in task deallocation (#5172) 2022-11-06 22:13:25 +01:00
Alice Ryhl
9884fe3394
runtime: fix unsync_load on atomic types (#5175) 2022-11-06 14:38:18 +01:00
Alice Ryhl
a1002a2203
ci: update miri flags (#5174) 2022-11-06 13:41:53 +01:00
Alice Ryhl
fc9518b627
chore: bump clippy version (#5173) 2022-11-06 12:44:26 +01:00
Yiyu Lin
f4643608ad
runtime: fix typo in expect message (#5169) 2022-11-05 11:18:27 +00:00
Carl Lerche
687aa2bae5
rt: move enter into context (#5167)
This moves the functions, types, and thread-local related to entering a
runtime into the context module. This does not yet unify the thread-local
variables, as that, will be done in a follow-up PR.
2022-11-04 15:08:51 -07:00
Carl Lerche
b2f5dbea47
rt: remove handle reference from each scheduler (#5166)
Instead of each scheduler flavor holding a reference to the scheduler
handle, the scheduler handle is passed in as needed. This removes a
duplicate handle reference in the `Runtime` struct and lays the
groundwork for further handle struct tweaks.
2022-11-04 13:36:12 -07:00
Duarte Nunes
23fdd32b01
rt: export metrics about the blocking thread pool (#5161)
Publish the blocking thread pool metrics as thread-safe values, written
under the blocking thread pool's lock and able to be read in a lock-free
fashion by any reader.

Fixes #5156
2022-11-04 11:05:36 -07:00
Hubert Hirtz
5b46395a1e
sync: fix Sync assertion for AtomicWaker (#5165) 2022-11-04 13:22:30 +01:00
Carl Lerche
32da1aa9da
rt: unify entering a runtime with Handle::enter (#5163)
This is a first step towards unifying the concepts of "entering a
runtime" and setting `Handle::current`.

Previously, these two operations were performed separately at each call
site (runtime block_on, ...). This is error-prone and also requires
multiple accesses to the thread-local variable. Additionally, "entering
the runtime" conflated the concept of entering a blocking region. For
example, calling `mpsc::Receiver::recv_blocking` performed the "enter
the runtime" step. This was done to prevent blocking a runtime, as the
operation will panic when called from an existing runtime.

To untangle these concepts, the patch splits out each logical operation
into functions. In total, there are three "enter" operations:

* `set_current_handle`
* `enter_runtime`
* `enter_blocking_region`

There are some behavior changes with each function, but they
should not translate to public behavior changes. The most significant is
`enter_blocking_region` does not change the value of the thread-local
variable, which means the function can be re-entered. Since
`enter_blocking_region` is an internal-only function and we do not
re-enter, this has no public-facing impact.

Because `enter_runtime` takes a `&Handle` to combine the
`set_current_handle` operation with entering a runtime, the patch
exposes an annoyance with the current `scheduler::Handle` struct layout.
A new instance of `scheduler::Handle` must be constructed at each call
to `enter_runtime`. We can explore cleaning this up later.

This patch also does not combine the "entered runtime" thread-local
variable with the "context" thread-local variable. To keep the patch
smaller, this has been punted to a follow-up change.
2022-11-03 15:01:08 -07:00
Carl Lerche
df6348fb4a
chore: fix tests for Rust 1.65 release (#5164)
Rust 1.65 reduces some struct sizes.
2022-11-03 14:24:10 -07:00
Rustom Shareef
74a29be607
chore: typo in TryLockError for RwLock::try_write (#5160)
Fixes an incoherent sentence describing when RwLock::try_write (and related) would error with TryLockError.
2022-11-02 18:13:31 -05:00
Carl Lerche
26791a62bc
rt: move Runtime into its own file (#5159)
This removes the `Runtime` implementation from a cfg macro so it can be
formatted.
2022-11-02 15:44:56 -07:00
Carl Lerche
467adec4e1
rt: move park logic into runtime module (#5158)
The runtime is the primary user of parking. Moving park into runtime
will help future cleanups around combining context entering w/ block_on
calls.
2022-11-02 14:37:44 -07:00
Carl Lerche
d8ee205530
rt: move budget state to context thread-local (#5157)
This patch consolidates the budget thread-local state with the
`runtime::context` thread local. This reduces the number of thread-local
variables used by Tokio by one.
2022-11-02 12:24:12 -07:00
Carl Lerche
ca8e176ce9
rt: rm coop::budget from LocalSet::run_until (#5155)
The `LocalSet::run_until` future is just a "plain" future that should
run on a runtime that already has a coop budget. In other words, the
`run_until` future should not get its own budget but should inherit the
calling task's budget. Getting this behavior is done by removing the
call to `budget` in `run_until`.
2022-11-01 14:03:01 -07:00
Carl Lerche
a051ed726f
rt: move coop mod into runtime (#5152)
This is a step towards unifying thread-local variables. In the future,
`coop` will be updated to use the runtime context thread-local to store
its state.
2022-11-01 09:03:56 -07:00
Yiyu Lin
203a079743
sync: make Notify panic safe (#5154) 2022-11-01 14:24:01 +01:00
Carl Lerche
b1f40f4356
rt: rename some confusing internal variables/fns (#5151)
This patch does some internal renames to remove some confusion.

* `allow_blocking` is renamed to `allow_block_in_place` to indicate that
  the variable only impacts the `block_in_place()` function.

* `context::try_enter` is renamed to `context::try_set_current` to
  disambiguate between the various "enter" functions. This function only
  sets the runtime handle used by Tokio's public APIs. Entering a runtime
  is a separate operation.  # Please enter the commit message for your
  changes.

* `scheduler::Handle::enter()` is removed to consolidate methods that
  set the current context.
2022-10-31 15:06:37 -07:00
Alice Ryhl
a9d5eb2fc7
io: add lines example for StreamReader (#5145) 2022-10-31 20:40:52 +01:00
Matt Schulte
c2210dfe37
net: fix function name in UdpSocket recv documentation (#5150)
In the "cancellation safety" section of the UdpSocket recv function, "recv_from" is referenced when it should be "recv"
2022-10-31 16:10:26 +00:00
Carl Lerche
df99428c17
rt: add runtime::context to unify thread-locals (#5143)
This patch is the first step towards unifying all the thread-local
variables spread out across Tokio. A new `Context` struct is added which
will be used to replace the various thread-locals that exist today.

Initially, `Context` only holds the current runtime handle and the
random number generator. Further PRs will add other thread-local state.

A previous PR removed `runtime::context`. At that time,
`runtime::context` was used as an extra layer to access the various
runtime driver handles. This version of `runtime::context` serves a
different purpose (unifying all the thread-locals).
2022-10-31 09:05:10 -07:00