3192 Commits

Author SHA1 Message Date
Carl Lerche
22cff80048
chore: update CI's clippy version to 1.65 (#5276) 2022-12-06 19:56:13 -08:00
Alan Somers
07da5e73ee
ci: update CI environment to FreeBSD 12.4 (#5272)
12.3 will soon be EoL
2022-12-06 10:03:51 +01:00
Alan Somers
c4ed16d1b4
ci: future-proof for FreeBSD 12 (#5260)
Raise the mio-aio dev dependency, which transitively brings in Nix, to
ensure that the tests will continue to compile if libc switches from a
FreeBSD 11 ABI to a FreeBSD 12 one.
2022-12-06 10:03:09 +01:00
Carl Lerche
3ce5a2681c
chore: prepare Tokio v1.23 release (#5270)
### Fixed
 - net: fix Windows named pipe connect ([#5208])
 - io: support vectored writes for `ChildStdin` ([#5216])
 - io: fix `async fn ready()` false positive for OS-specific events ([#5231])

 ### Changed
 - runtime: `yield_now` defers task until after driver poll ([#5223])
 - runtime: reduce amount of codegen needed per spawned task ([#5213])
 - windows: replace `winapi` dependency with `windows-sys` ([#5204])

 [#5208]: https://github.com/tokio-rs/tokio/pull/5208
 [#5216]: https://github.com/tokio-rs/tokio/pull/5216
 [#5213]: https://github.com/tokio-rs/tokio/pull/5213
 [#5204]: https://github.com/tokio-rs/tokio/pull/5204
 [#5223]: https://github.com/tokio-rs/tokio/pull/5223
 [#5231]: https://github.com/tokio-rs/tokio/pull/5231
tokio-1.23.0
2022-12-05 15:22:43 -08:00
Tymoteusz Wiśniewski
644cb8207d
rt: fix *_closed false positives (#5231)
Readiness futures inconsistently return the current readiness of an I/O resource if it is immediately available, or all readiness relevant for the given `Interest`, if a future needs to wait. In particular, it always returns `read_closed` for `Interest::READABLE` and `write_closed` for `Interest::WRITABLE`, which often is not true. Tokio should not tolerate false positives for `*_closed` events because they are considered final states and are not cleared internally.

In the case of an `io_resource.ready(Interest::READABLE | Interest::WRITABLE)` call, this behavior may also lead to false positives of other events.

## Solution

Follow the same strategy as `poll_ready` and return the current resource's readiness.

Closes: #5098
2022-12-05 14:42:49 -08:00
Jiahao XU
a1316cd792
io: impl std::io::BufRead on SyncIoBridge<T> (#5265)
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-12-05 10:11:44 +01:00
sharnoff
86ffabe2af
docs: add note about current-thread + Handle::block_on (#5264)
There's already an existing warning about this combo in the
documentation for `Handle::block_on`. This commit adds a summarized
version in `Runtime::handle`.
2022-12-05 00:14:05 -06:00
Vitalii Kryvenko
00bf5ee8a8
sync: improve watch docs (#5261) 2022-12-04 00:06:12 +00:00
Tilman
87510100ce
Fix typo (#5255) 2022-12-03 10:49:55 +00:00
Carl Lerche
2be71ad746
chore: move conditional AtomicU64 impl to new file (#5256)
Keeping the implementation out of a macro lets rustfmt apply to it.
2022-12-02 11:47:11 -08:00
Carl Lerche
d1b789f33a
rt: fix new yield_now behavior with block_in_place (#5251)
PR #5223 changed the behavior of `yield_now()` to store yielded tasks
and notify them *after* polling the resource drivers. This PR fixes a
couple of bugs with this new behavior when combined with
`block_in_place()`.

First, we need to avoid freeing the deferred task queue when exiting a
runtime if it is *not* the root runtime. Because `block_in_place()`
allows a user to start a new runtime from within an existing task, this
check is necessary.

Second, when a worker core is stolen from a thread during a
`block_in_place()` call, we need to ensure that deferred tasks are
notified anyway.
2022-12-01 17:23:33 -08:00
Carl Lerche
22862739dd
rt: yield_now defers task until after driver poll (#5223)
Previously, calling `task::yield_now().await` would yield the current
task to the scheduler, but the scheduler would poll it again before
polling the resource drivers. This behavior can result in starving the
resource drivers.

This patch creates a queue tracking yielded tasks. The scheduler
notifies those tasks **after** polling the resource drivers.

Refs: #5209
2022-11-30 14:21:08 -08:00
Taiki Endo
993a60b7c7
chore: prepare tokio-macros v1.8.2 (#5246) tokio-macros-1.8.2 2022-11-30 22:10:13 +09:00
Taiki Endo
5d25ec46d5
macros: fix span of body variable (#5244) 2022-11-30 03:50:41 +09:00
Thomas de Zeeuw
766f22fae3 Prepare tokio-macros 1.8.1 tokio-macros-1.8.1 2022-11-29 15:08:09 +00:00
Thomas de Zeeuw
f5686f6bc0 Use #crate_ident in test macro
Instead of ::tokio.
2022-11-29 14:28:13 +00:00
Thomas de Zeeuw
224acd2500 Pin Future to stack in #[tokio::test]
Instead of boxing it.
2022-11-29 14:28:13 +00:00
Thomas de Zeeuw
2fcc6c2cb0 Box Futures in #[tokio::test]
This reduces the amount of copies of the Runtime::block_on and related
functions the compiler has to generate and LLVM process. We've seen it
reduce the compilation time of our tests (some 1900 of them) from 40s
down to 12s, with no impact on the runtime of tests.

Below is an output of llvm-lines for our tests.

Before:

  Lines                  Copies                Function name
  -----                  ------                -------------
  8954414                156577                (TOTAL)
   984626 (11.0%, 11.0%)   9289 (5.9%,  5.9%)  std:🧵:local::LocalKey<T>::try_with
   648093 (7.2%, 18.2%)    1857 (1.2%,  7.1%)  tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
   557100 (6.2%, 24.5%)    3714 (2.4%,  9.5%)  tokio::park:🧵:CachedParkThread::block_on
   551679 (6.2%, 30.6%)    7430 (4.7%, 14.2%)  tokio::coop::with_budget::{{closure}}
   514389 (5.7%, 36.4%)    3714 (2.4%, 16.6%)  tokio::runtime::scheduler::current_thread::Context::enter
   326832 (3.6%, 40.0%)    1857 (1.2%, 17.8%)  tokio::runtime::scheduler::current_thread::CurrentThread::block_on
   291549 (3.3%, 43.3%)    1857 (1.2%, 19.0%)  tokio::runtime::scheduler::current_thread::CoreGuard::enter
   261907 (2.9%, 46.2%)    7430 (4.7%, 23.7%)  tokio::coop::budget
   189468 (2.1%, 48.3%)    7430 (4.7%, 28.5%)  tokio::coop::with_budget
   137418 (1.5%, 49.8%)    3714 (2.4%, 30.8%)  tokio::runtime::enter::Enter::block_on
   126276 (1.4%, 51.3%)    1857 (1.2%, 32.0%)  tokio::runtime::Runtime::block_on
   124419 (1.4%, 52.6%)    1857 (1.2%, 33.2%)  tokio::macros::scoped_tls::ScopedKey<T>::set
   118897 (1.3%, 54.0%)    3715 (2.4%, 35.6%)  core::option::Option<T>::or_else
   111420 (1.2%, 55.2%)    1857 (1.2%, 36.8%)  tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
   109408 (1.2%, 56.4%)    2105 (1.3%, 38.1%)  <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   105893 (1.2%, 57.6%)    9289 (5.9%, 44.0%)  std:🧵:local::LocalKey<T>::with
    96564 (1.1%, 58.7%)    1857 (1.2%, 45.2%)  tokio::runtime::scheduler::current_thread::Context::run_task
    90993 (1.0%, 59.7%)    7428 (4.7%, 50.0%)  tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
    90515 (1.0%, 60.7%)    2105 (1.3%, 51.3%)  core::pin::Pin<&mut T>::map_unchecked_mut
    89136 (1.0%, 61.7%)    1857 (1.2%, 52.5%)  tokio::runtime::scheduler::multi_thread::MultiThread::block_on

After:

  Lines                  Copies               Function name
  -----                  ------               -------------
  3188618                41634                (TOTAL)
   109408 (3.4%,  3.4%)   2105 (5.1%,  5.1%)  <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
    90515 (2.8%,  6.3%)   2105 (5.1%, 10.1%)  core::pin::Pin<&mut T>::map_unchecked_mut
    56220 (1.8%,  8.0%)   1874 (4.5%, 14.6%)  alloc::boxed::Box<T>::pin
    48333 (1.5%,  9.5%)   2179 (5.2%, 19.8%)  core::ops::function::FnOnce::call_once
    28587 (0.9%, 10.4%)      1 (0.0%, 19.8%)  XXXXXXXXXXXXXXXXXXX
    18730 (0.6%, 11.0%)   1873 (4.5%, 24.3%)  alloc::boxed::Box<T,A>::into_pin
    16190 (0.5%, 11.5%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    15870 (0.5%, 12.0%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    15250 (0.5%, 12.5%)      1 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXX
    12801 (0.4%, 12.9%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12801 (0.4%, 13.3%)      2 (0.0%, 24.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12630 (0.4%, 13.7%)   2105 (5.1%, 29.4%)  <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::{{closure}}
    12613 (0.4%, 14.1%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12613 (0.4%, 14.5%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12613 (0.4%, 14.9%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    12613 (0.4%, 15.3%)      2 (0.0%, 29.4%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    11395 (0.4%, 15.7%)     96 (0.2%, 29.7%)  alloc::alloc::box_free
    11364 (0.4%, 16.0%)   1891 (4.5%, 34.2%)  <T as core::convert::Into<U>>::into
    11238 (0.4%, 16.4%)   1873 (4.5%, 38.7%)  alloc::boxed::<impl core::convert::From<alloc::boxed::Box<T,A>> for core::pin::Pin<alloc::boxed::Box<T,A>>>::from
    10735 (0.3%, 16.7%)      2 (0.0%, 38.7%)  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Note that I have replaced our test functions with XXX. As you can
clearly see they're not in the top 20 in the before output, while
they're in the after oput.

Further note that the amount of copies have been reduced from 156577 to
41634.
2022-11-29 14:28:13 +00:00
Tymoteusz Wiśniewski
28ec4a6161
net: update try_io docs with interest limitations (#5222) 2022-11-28 10:38:40 +01:00
Loong Dai
939b5bb42f
tests: fix a typo (#5236)
Signed-off-by: Loong <loong.dai@intel.com>
2022-11-28 10:37:22 +01:00
Abutalib Aghayev
718d6ce8ca
task: fix the incomplete/wrong description of JOIN_WAKER bit. (#5217)
* task: fix the incomplete/wrong description of JOIN_WAKER bit (#5217)
2022-11-24 15:47:15 -05:00
Tymoteusz Wiśniewski
e316428210
net: replace socket with pipe in named pipe docs (#5221) 2022-11-23 13:48:30 +01:00
Taiki Endo
fc83e01949
chore: add tokio_no_atomic_u64 cfg (#5226) 2022-11-23 19:00:57 +09:00
Loong Dai
3a5f7b7f2f
examples: update hello world comment (#5219) 2022-11-23 11:00:25 +01:00
Loong Dai
6da81471f9
signal: fix a typo (#5224) 2022-11-23 10:59:44 +01:00
Kenny Kerr
299bd6aee3
net: replace winapi with windows-sys (#5204) 2022-11-22 16:23:26 +00:00
Loong Dai
e14307393a
runtime: fix typo in runtime builder docs (#5218)
Signed-off-by: Loong <loong.dai@intel.com>
2022-11-22 08:24:12 +00:00
Alice Ryhl
45e37dbfa2
runtime: reduce codegen per task (#5213)
This PR should hopefully reduce the amount of code generated per
future-type spawned on the runtime. The following methods are no longer generic:

* `try_set_join_waker`
* `remote_abort`
* `clone_waker`
* `drop_waker`
* `wake_by_ref`
* `wake_by_val`

A new method is added to the vtable called schedule, which is used when a task
should be scheduled on the runtime. E.g. wake_by_ref will call it if the state change
says that the task needs to be scheduled. However, this method is only generic over
the scheduler, and not the future type, so it also isn't generated for every task.

Additionally, one of the changes involved in the above makes it possible to remove
the id field from JoinHandle and AbortHandle.
2022-11-21 15:38:22 -08:00
Jiahao XU
304b5152a7
sync: add owned future for CancellationToken (#5153) 2022-11-21 15:03:09 +00:00
Joonas Koivunen
f15d14ee91
process: support ChildStdin::poll_write_vectored on unix (#5216) 2022-11-21 13:57:24 +00:00
Alice Ryhl
6a2cd9a652
doc: update parking_lot info in feature list (#5215) 2022-11-21 13:42:55 +01:00
Tymoteusz Wiśniewski
2682c505e8
net: fix named pipe connect (#5208) 2022-11-20 21:02:13 +00:00
Carl Lerche
808d52563e
ci: run tests on ARM and i686 using cross (#5196)
This patch updates CI to use `cross` to run Tokio tests on virtualized
ARM and i686 VMs. Because ipv6 doesn't work on Github action running in
a docker instance, those tests are disabled
2022-11-18 14:02:53 -08:00
Alice Ryhl
bf31759bff
chore: prepare Tokio v1.22.0 (#5203) tokio-1.22.0 2022-11-18 13:15:06 -08:00
John DiSanti
d65826236b
ci: remove libc types from external types allow list (#5197) 2022-11-17 11:57:55 +01:00
Lucas Kent
1cbbcc9ad5
sync: specify return type of oneshot::Receiver in docs (#5198) 2022-11-17 11:56:56 +01:00
Abutalib Aghayev
a668020150
net: remove libc type leakage in a public API (#5191) 2022-11-15 23:28:33 +01:00
Alice Ryhl
01f0193971
chore: fix compilation on master (#5190) 2022-11-13 13:21:08 +00:00
Abutalib Aghayev
71bd49e146
task: add task::id() and task::try_id() (#5171) 2022-11-13 14:18:42 +01:00
Artyom Kozhemiakin
582d512907
sync: add mpsc::WeakUnboundedSender (#5189)
Signed-off-by: Artyom Kozhemiakin <xenus.t@gmail.com>
2022-11-12 22:29:13 +00:00
Carl Lerche
b7812c85ca
rt: fix LocalSet drop in thread local (#5179)
`LocalSet` cleans up any tasks that have not yet been completed when it is
dropped. Previously, this cleanup process required access to a thread-local.
Suppose a `LocalSet` is stored in a thread-local itself. In that case, when it is
dropped, there is no guarantee the drop implementation will be able to
access the internal `LocalSet` thread-local as it may already have been
destroyed.

The internal `LocalSet` thread local is mainly used to avoid writing unsafe
code. All `LocalState` that cannot be moved across threads is stored in the
thread-local and accessed on demand.

This patch moves this local-only state into the `LocalSet`'s "shared" struct.
Because this struct *is* `Send`, the local-only state is stored in `UnsafeCell`,
and callers must ensure not to touch it from other threads.

A debug assertion is added to enforce this requirement in tests.

Fixes #5162
2022-11-10 10:06:25 -08:00
Abutalib Aghayev
9e3fb1673a
rt: move CoreStage methods to Core (#5182) 2022-11-10 13:15:30 +00:00
Carl Lerche
53cba023da
rt: fix accidental unsetting of current handle (#5178)
An earlier change updated `enter_runtime` to also set the current
handle. However, the change did not store the `SetCurrentGuard`, so the
"current handle" was immediately unset. This patch stores the
`SetCurrentGuard` in the `EnterRuntimeGuard`.

No existing test exposed this bug because all tests went via `Runtime`
instead of `Handle`. Currently, `Runtime` is still explicitly setting
the handle before entering runtime, so all tests still passed. A new
test is added that covers the case of calling `Handle::block_on` and
accessing the current handle.
2022-11-09 12:13:30 -08:00
Carl Lerche
236d026667
rt: combine context and entered thread-locals (#5168)
A previous patch moved code related to entering a runtime into the
context module but did not change anything. This patch combines both
thread-local variables.
2022-11-07 13:35:54 -08:00
Alice Ryhl
909439c9f5
task: elaborate safety comments in task deallocation (#5172) 2022-11-06 22:13:25 +01:00
Alice Ryhl
9884fe3394
runtime: fix unsync_load on atomic types (#5175) 2022-11-06 14:38:18 +01:00
Alice Ryhl
a1002a2203
ci: update miri flags (#5174) 2022-11-06 13:41:53 +01:00
Alice Ryhl
fc9518b627
chore: bump clippy version (#5173) 2022-11-06 12:44:26 +01:00
Yiyu Lin
f4643608ad
runtime: fix typo in expect message (#5169) 2022-11-05 11:18:27 +00:00
Carl Lerche
687aa2bae5
rt: move enter into context (#5167)
This moves the functions, types, and thread-local related to entering a
runtime into the context module. This does not yet unify the thread-local
variables, as that, will be done in a follow-up PR.
2022-11-04 15:08:51 -07:00