Carl Lerche 4eed411519
rt: reduce no-op wakeups in the multi-threaded scheduler (#4383)
This patch reduces the number of times worker threads wake up without having
work to do in the multi-threaded scheduler. Unnecessary wake-ups are expensive
and slow down the scheduler. I have observed this change reduce no-op wakes
by up to 50%.

The multi-threaded scheduler is work-stealing. When a worker has tasks to process,
and other workers are idle (parked), these idle workers must be unparked so that
they can steal work from the busy worker. However, unparking threads is expensive,
so there is an optimization that avoids unparking a worker if there already exists
workers in a "searching" state (the worker is unparked and looking for work). This
works pretty well, but transitioning from 1 "searching" worker to 0 searching workers
introduces a race condition where a thread unpark can be lost:

* thread 1: last searching worker about to exit searching state
* thread 2: needs to unpark a thread, but skip because there is a searching worker.
* thread 1: exits searching state w/o seeing thread 2's work.

Because this should be a rare condition, Tokio solves this by always unparking a
new worker when the current worker:

* is the last searching worker
* is transitioning out of searching
* has work to process.

When the newly unparked worker wakes, if the race condition described above
happened, "thread 2"'s work will be found. Otherwise, it will just go back to sleep.

Now we come to the issue at hand. A bug incorrectly set a worker to "searching"
when the I/O driver unparked the thread. In a situation where the scheduler was
only partially under load and is able to operate with 1 active worker, the I/O driver
would unpark the thread when new I/O events are received, incorrectly transition
it to "searching", find new work generated by inbound I/O events, incorrectly
transition itself from the last searcher -> no searchers, and unpark a new thread.
This new thread would wake, find no work and go back to sleep.

Note that, when the scheduler is fully saturated, this change will make no impact
as most workers are always unparked and the optimization to avoid unparking
threads described at the top apply.
2022-01-13 15:18:32 -08:00
2022-01-13 00:27:05 +09:00
2021-11-23 12:09:24 +09:00

Tokio

A runtime for writing reliable, asynchronous, and slim applications with the Rust programming language. It is:

  • Fast: Tokio's zero-cost abstractions give you bare-metal performance.

  • Reliable: Tokio leverages Rust's ownership, type system, and concurrency model to reduce bugs and ensure thread safety.

  • Scalable: Tokio has a minimal footprint, and handles backpressure and cancellation naturally.

Crates.io MIT licensed Build Status Discord chat

Website | Guides | API Docs | Chat

Overview

Tokio is an event-driven, non-blocking I/O platform for writing asynchronous applications with the Rust programming language. At a high level, it provides a few major components:

  • A multithreaded, work-stealing based task scheduler.
  • A reactor backed by the operating system's event queue (epoll, kqueue, IOCP, etc...).
  • Asynchronous TCP and UDP sockets.

These components provide the runtime components necessary for building an asynchronous application.

Example

A basic TCP echo server with Tokio.

Make sure you activated the full features of the tokio crate on Cargo.toml:

[dependencies]
tokio = { version = "1.15.0", features = ["full"] }

Then, on your main.rs:

use tokio::net::TcpListener;
use tokio::io::{AsyncReadExt, AsyncWriteExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let listener = TcpListener::bind("127.0.0.1:8080").await?;

    loop {
        let (mut socket, _) = listener.accept().await?;

        tokio::spawn(async move {
            let mut buf = [0; 1024];

            // In a loop, read data from the socket and write the data back.
            loop {
                let n = match socket.read(&mut buf).await {
                    // socket closed
                    Ok(n) if n == 0 => return,
                    Ok(n) => n,
                    Err(e) => {
                        eprintln!("failed to read from socket; err = {:?}", e);
                        return;
                    }
                };

                // Write the data back
                if let Err(e) = socket.write_all(&buf[0..n]).await {
                    eprintln!("failed to write to socket; err = {:?}", e);
                    return;
                }
            }
        });
    }
}

More examples can be found here. For a larger "real world" example, see the mini-redis repository.

To see a list of the available features flags that can be enabled, check our docs.

Getting Help

First, see if the answer to your question can be found in the Guides or the API documentation. If the answer is not there, there is an active community in the Tokio Discord server. We would be happy to try to answer your question. You can also ask your question on the discussions page.

Contributing

🎈 Thanks for your help improving the project! We are so happy to have you! We have a contributing guide to help you get involved in the Tokio project.

In addition to the crates in this repository, the Tokio project also maintains several other libraries, including:

  • hyper: A fast and correct HTTP/1.1 and HTTP/2 implementation for Rust.

  • tonic: A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility.

  • warp: A super-easy, composable, web server framework for warp speeds.

  • tower: A library of modular and reusable components for building robust networking clients and servers.

  • tracing (formerly tokio-trace): A framework for application-level tracing and async-aware diagnostics.

  • rdbc: A Rust database connectivity library for MySQL, Postgres and SQLite.

  • mio: A low-level, cross-platform abstraction over OS I/O APIs that powers tokio.

  • bytes: Utilities for working with bytes, including efficient byte buffers.

  • loom: A testing tool for concurrent Rust code

Supported Rust Versions

Tokio is built against the latest stable release. The minimum supported version is 1.46. The current Tokio version is not guaranteed to build on Rust versions earlier than the minimum supported version.

Release schedule

Tokio doesn't follow a fixed release schedule, but we typically make one to two new minor releases each month. We make patch releases for bugfixes as necessary.

Bug patching policy

For the purposes of making patch releases with bugfixes, we have designated certain minor releases as LTS (long term support) releases. Whenever a bug warrants a patch release with a fix for the bug, it will be backported and released as a new patch release for each LTS minor version. Our current LTS releases are:

  • 1.8.x - LTS release until February 2022.
  • 1.14.x - LTS release until June 2022.

Each LTS release will continue to receive backported fixes for at least half a year. If you wish to use a fixed minor release in your project, we recommend that you use an LTS release.

To use a fixed minor version, you can specify the version with a tilde. For example, to specify that you wish to use the newest 1.8.x patch release, you can use the following dependency specification:

tokio = { version = "~1.8", features = [...] }

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Tokio by you, shall be licensed as MIT, without any additional terms or conditions.

Description
A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...
Readme MIT 75 MiB
Languages
Rust 100%