This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.
This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans.
Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.
Before:
```
test enter_span ... bench: 3 ns/iter (+/- 0)
test span_no_fields ... bench: 51 ns/iter (+/- 12)
test span_repeatedly ... bench: 5,073 ns/iter (+/- 1,528)
test span_with_fields ... bench: 56 ns/iter (+/- 49)
test span_with_fields_record ... bench: 363 ns/iter (+/- 61)
```
After:
```
test enter_span ... bench: 3 ns/iter (+/- 0)
test span_no_fields ... bench: 35 ns/iter (+/- 12)
test span_repeatedly ... bench: 4,165 ns/iter (+/- 298)
test span_with_fields ... bench: 48 ns/iter (+/- 12)
test span_with_fields_record ... bench: 363 ns/iter (+/- 91)
```
Closes#1032
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
on Rust 2018
* Fix an example in the documentation
## Motivation
To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.
## Solution
A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.
`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.
New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.
Fixes: #986Closes: #1008
Co-Authored-By: csmoe <csmoe@msn.com>
## Motivation
Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.
However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`. Even when filters change very
infrequently, they must still always be re-evaluated every time.
In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.
## Solution
This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.
## Breaking Change
In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.
Closes#1038
Co-Authored-By: yaahallo <jlusby42@gmail.com>
## Motivation
The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.
Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
span.record("bar" &false);
});
```
Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.
The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.
## Solution
This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows.
In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`.
## Notes
In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.
Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.
Before:
```
test enter_span ... bench: 13 ns/iter (+/- 1)
```
...and after:
```
test enter_span ... bench: 3 ns/iter (+/- 1)
```
Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.
I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.
## Solution
This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.
## Notes
The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.
Closes#1018.
Closes#1022.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952).
Closes#1013
## Solution
Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels
## Notes
I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
when calling `global_init`
Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS
Fixes#999
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.
Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.
When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.
The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.
Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.
## Solution
This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation.
## Notes
In order to allow spans to log their names when they are entered and
exited even when the span is disabled, this branch adds an
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.
Fixes: #949
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.
## Solution
We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.
Closes#959
## Motivation
In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to
downcast the current `Dispatch` to a concrete subscriber type.
For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust
pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
tokio_trace::dispatcher::with(|c| {
if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
s.do_elizatracing_correlation_magic(tt, et);
}
});
}
```
This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).
## Solution
This branch makes the following changes:
* Add an object-safe `downcast_raw` method to the `Subscriber` trait,
taking a `TypeId` and returning an `*const ()` if the type ID
matches the subscriber's type ID, or `None` if it does not, and
* Add `is<T>` and `downcast_ref<T>` functions to `Subscriber`
and `Dispatch`, using `downcast_raw`.
Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour
somehow, can downcast to the inner type if they choose to.
[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204
Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
* chore: Fix examples not working with `cargo run`
## Motivation
PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:
```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜ cargo run --example chat
error: no example target named `chat`
Did you mean `echo`?
```
## Solution
This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:
```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜ cargo run --example chat
Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
Compiling tokio-reactor v0.1.9
Compiling tokio-threadpool v0.1.13
Compiling tokio-current-thread v0.1.6
Compiling tokio-timer v0.2.10
Compiling tokio-uds v0.2.5
Compiling tokio-udp v0.1.3
Compiling tokio-tcp v0.1.3
Compiling tokio-fs v0.1.6
Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
Finished dev [unoptimized + debuginfo] target(s) in 7.04s
Running `target/debug/examples/chat`
server running on localhost:6142
```
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.
When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.
This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
This branch makes the following changes to `tokio-trace`'s `Span` type:
* **Remove manual close API from spans**
In practice, there wasn't really a use-case for this, and it
complicates the implementation a bit. We can always add it back later.
* **Remove generic lifetime from `Span`**
Again, there wasn't actually a use-case for spans with metadata that
doesn't live for the static lifetime, and it made using `Span`s in
other types somewhat inconvenient. It's also possible to implement an
alternative API for non-static spans on top of the `tokio-trace-core`
primitives.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.
This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.
Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.
Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
