This branch adds some documentation to the `FormatEvent` trait in
`tracing_subscriber::fmt` on how to write user-provided `FormatEvent`
implementations. There's probably room for additional improvement here,
but I just wanted to get something written down for now.
I also fixed a broken link I noticed while I was here.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
This branch adds some documentation to the `FormatEvent` trait in
`tracing_subscriber::fmt` on how to write user-provided `FormatEvent`
implementations. There's probably room for additional improvement here,
but I just wanted to get something written down for now.
I also fixed a broken link I noticed while I was here.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
Depends on #1696
## Motivation
PR #1696 adds a new API for propagating whether or not ANSI formatting
escape codes are enabled to the event formatter and field formatter via
the `Writer` type. This means that it's now quite easy to make field
formatting also include ANSI formatting escape codes when ANSI
formatting is enabled. Currently, `tracing-subscriber`'s default field
formatter does not use ANSI formatting --- ANSI escape codes are
currently only used for parts of the event log line controlled by the
event formatter, not within fields. Adding ANSI colors and formatting in
the fields could make the formatted output nicer looking and easier to
read.
## Solution
This branch adds support for ANSI formatting escapes in the default
field formatter, when ANSI formatting is enabled. Now, field names will
be italicized, and the `=` delimiter is dimmed. This should make it a
little easier to visually scan the field lists, since the names and
values are more clearly separated and should be easier to distinguish.
Additionally, I cleaned up the code for conditionally using ANSI
formatting significantly. Now, the `Writer` type has (crate-private)
methods for returning `Style`s for bold, dimmed, and italicized text,
when ANSI escapes are enabled, or no-op `Style`s when they are not. This
is reused in all the formatter implementations, so it makes sense to
have it in one place. I also added dimmed formatting of delimiters in a
couple other places in the default event format, which I thought
improved readability a bit by bringing the actual *text* to the
forefront.
Example of the default format with ANSI escapes enabled:
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
This backports PR #1696 to v0.1.x.
## Motivation
Currently, whether `tracing-subscriber`'s `fmt` subscriber will ANSI
formatting escape codes use is configured on the `Format` type. This
means that the configuration is honored by the event formatters
implemented in `tracing-subscriber`, but is not easily exposed to those
in other crates. Additionally, it's not currently easy to expose the
configuration to the field formatter, so it's difficult to implement
field formatters that use ANSI escape codes conditionally.
Issue #1651 suggested a new API for this, where the writer that's passed
in to the event and field formatters provides a method for checking if
ANSI escape codes are supported.
## Solution
This branch adds a new method to the `Writer` type added in #1661. The
`FormatEvent` and `FormatFields` implementations can call
`Writer::has_ansi_escapes` to determine whether ANSI escape codes are
supported. This is also propagated to `FormattedFields`, so that it can
be determined when adding new fields to a preexisting set of formatted
fields.
Fixes#1651
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
In `tracing-subscriber` 0.3.x, `MakeWriter` filtering seems to have
stopped working when using the `BoxMakeWriter` wrapper to erase the type
of a `MakeWriter` implementation. It looks like what happened is that
commit 6cc6c47354ceeb47da7c95faa41c6d29b71b5f37, which backported the
change to add a lifetime parameter to `MakeWriter` (#781), I
accidentally clobbered the `make_writer_for` method on the inner `Boxed`
type, so that it only has `make_writer`:
6cc6c47354 (diff-c5dc275b15a60c1a2d4694da3797f4247c4f2e1e0978fd210dd14452d6746283L737-L739)
This meant that any filtering performed by the `MakeWriter` inside the
box is now ignored. My bad!
## Solution
This commit puts back the missing `make_writer_for` method. Whoops!
Fixes#1694
# 0.3.1 (Oct 25, 2021)
This release fixes a few issues related to feature flagging.
### Fixed
- **time**: Compilation error when enabling the "time" feature flag
without also enabling the "local-time" feature flag ([#1685])
- **registry**: Unused method warnings when the "std" feature is enabled
but the "registry" feature is disabled ([#1686])
[#1685]: https://github.com/tokio-rs/tracing/pull/1685
[#1686]: https://github.com/tokio-rs/tracing/pull/1686
## Motivation
Currently, there are some `pub(crate)` methods on the `Extensions`,
`ExtensionsMut`, and `ExtensionsInner` types that are only used when the
"registry" feature is enabled. However, the _types_ exist when the "std"
feature flag is enabled, so those methods produce unused method
warnings.
## Solution
This commit adds `#[cfg(feature = "registry")]` to those methods, fixing
the warning.
## Motivation
Currently, the `pub use` re-export for the `LocalTime` struct is enabled
whenever the "time" feature flag is enabled. However, this type only
_exists_ when the "local-time" feature flag is enabled, so enabling only
the `time` feature results in a compilation error.
## Solution
This commit adds a separate `pub use` for `LocalTime` that's only
enabled when the "local-time" feature is enabled.
Fixes#1683
# 0.3.0 (Oct 22, 2021)
This is a breaking release of `tracing-subscriber`. The primary breaking
change in this release is the removal of the dependency on the [`chrono`
crate], due to [RUSTSEC-2020-0159]. To replace `chrono`, support is
added for formatting timestamps using the [`time` crate] instead.
In addition, this release includes a number of other breaking API
changes, such as adding (limited) support for `#![no_std]` targets,
removing previously deprecated APIs, and more.
### Breaking Changes
- Removed APIs deprecated in the v0.2.x release series.
- Renamed `Layer::new_span` to `Layer::on_new_span` ([#1674])
- Removed `Layer` impl for `Arc<L: Layer<S>>` and `Arc<dyn Layer<S> +
...>` ([#1649])
- Replaced the [`chrono` crate] with the [`time` crate] for timestamp
formatting, to resolve [RUSTSEC-2020-0159] ([#1646])
- Removed `json` and `env-filter` from default features. They must now
be enabled explictly ([#1647])
- Changed `FormatEvent::format_event` and `FormatFields::format_fields`
trait methods to take a `Writer` type, rather than a `&mut dyn
fmt::Write` trait object ([#1661])
- Changed the signature of the `MakeWriter` trait by adding a lifetime
parameter ([#781])
### Changed
- **layer**: Renamed `Layer::new_span` to `Layer::on_new_span` ([#1674])
- **fmt**: Changed `FormatEvent::format_event` and
`FormatFields::format_fields` trait methods to take a `Writer` type,
rather than a `&mut dyn fmt::Write` trait object ([#1661])
- **json**, **env-filter**: `json` and `env-filter` feature flags are no
longer enabled by default ([#1647])
### Removed
- Removed deprecated `CurrentSpan` type ([#1320])
- **registry**: Removed deprecated `SpanRef::parents` iterator, replaced
by `SpanRef::scope` in [#1431] ([#1648)])
- **layer**: Removed deprecated `Context::scope` iterator, replaced by
`Context::span_scope` and `Context::event_scope` in [#1431] and
[#1434] ([#1648)])
- **layer**: Removed `Layer` impl for `Arc<L: Layer<S>>` and `Arc<dyn
Layer<S> + ...>`. These interfere with per-layer filtering. ([#1649])
- **fmt**: Removed deprecated `LayerBuilder` type ([#1673])
- **fmt**: Removed `fmt::Layer::on_event` (renamed to
`fmt::Layer::fmt_event`) ([#1673])
- **fmt**, **chrono**: Removed the `chrono` feature flag and APIs for
using the [`chrono` crate] for timestamp formatting ([#1646])
### Added
- **fmt**, **time**: `LocalTime` and `UtcTime` types for formatting
timestamps using the [`time` crate] ([#1646])
- **fmt**: Added a lifetime parameter to the `MakeWriter` trait,
allowing it to return a borrowed writer. This enables implementations
of `MakeWriter` for types such as `Mutex<T: io::Write>` and
`std::fs::File`. ([#781])
- **env-filter**: Documentation improvements ([#1637])
- Support for some APIs on `#![no_std]` targets, by disabling the `std`
feature flag ([#1660])
Thanks to @Folyd and @nmathewson for contributing to this release!
[#1320]: https://github.com/tokio-rs/tracing/pull/1320
[#1673]: https://github.com/tokio-rs/tracing/pull/1673
[#1674]: https://github.com/tokio-rs/tracing/pull/1674
[#1646]: https://github.com/tokio-rs/tracing/pull/1646
[#1647]: https://github.com/tokio-rs/tracing/pull/1647
[#1648]: https://github.com/tokio-rs/tracing/pull/1648
[#1649]: https://github.com/tokio-rs/tracing/pull/1649
[#1660]: https://github.com/tokio-rs/tracing/pull/1660
[#1661]: https://github.com/tokio-rs/tracing/pull/1661
[#1431]: https://github.com/tokio-rs/tracing/pull/1431
[#1434]: https://github.com/tokio-rs/tracing/pull/1434
[#781]: https://github.com/tokio-rs/tracing/pull/781
[`chrono` crate]: https://crates.io/crates/chrono
[`time` crate]: https://crates.io/crates/time
[RUSTSEC-2020-0159]: https://rustsec.org/advisories/RUSTSEC-2020-0159.html
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
`tracing-subscriber` currently contains some APIs that were deprecated
in the v0.2.x series:
- `fmt::LayerBuilder`, which is now a type alias for `fmt::Layer` (as
the `Layer` type can expose all the same methods as the builder)
- `registry::SpanRef::parent_id`, which doesn't play nice with per-layer
filtering,
- `fmt::Layer::inherit_fields`, which no longer does anything as it's
now the default behavior
- `fmt::Layer::on_event`, which was renamed to `fmt_event`
- the `SpanRef::parents` and `layer::Context::scope` iterators, which
were replaced by the APIs added in #1431 and #1434
Prior to releasing v0.3, the deprecated APIs should be removed.
## Solution
This branch deletes the deprecated APIs, with the exception of
`SpanRef::parents` and `Context::scope` (which were already removed in
240d11a7ef70c1197f8071d98db74a02bd58fad9).
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Turns out that if `std` is enabled but `smallvec` is not, we have some
missing imports. This fixes the build when a crate passes
`default-features = "false` but then enables `std`.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Currently, the `FormatEvent` and `FormatFields` traits in
`tracing-subscriber` are passed a `&mut dyn fmt::Write` trait object to
write formatted representations of events and fields to. This is fine,
but it doesn't give us the ability to easily provide additional
configuration to the formatter, such as whether ANSI color codes are
supported.
Issue #1651 describes some approaches for adding a way to expose the
ANSI color code configuration to custom formatters. In particular, the
proposed solution involves wrapping the `fmt::Write` trait object in an
opaque struct, which can then implement additional methods for exposing
information like "are ANSI colors enabled" to the formatter. Since this
changes the signature of the `FormatEvent::format_event` and
`FormatFields::format_fields` methods, it's a breaking change.
Therefore, we need to make this change _now_ if we want to get the API
change in for `tracing-subscriber` 0.3.
## Solution
This branch adds a `Writer` struct that wraps the `&mut dyn fmt::Write`
trait object, and changes the various method signatures as appropriate.
It does **not** actually implement the change related to ANSI color
formatting. Once we change these methods' signatures to accept a
`Writer` struct, we can add as many methods to that struct as we like
without making additional breaking API changes. Therefore, this branch
_just_ makes the breaking change that's necessary to get in before v0.3
is released.
Propagating the ANSI color configuration can be implemented in a future
branch.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Currently, some `tracing-subscriber` dependencies in other crates use
`default-features = false`, to ensure that unneeded default features of
`tracing-subscriber` are not inadvertently enabled. However, some of the
features those crates enable also require the `std` feature flag. The
`default-features = false` config _disables_ the `std` feature, so the
required features are not available.
## Solution
This branch changes `tracing-subscriber`'s `fmt`, `registry`, and
`env-filter` feature flags to also enable the `std` feature flag
automatically.
## Alternatives
As an alternative solution, we could change crates that depend on
`tracing-subscriber` with `default-features = false` to also explicitly
ensure that the `std` feature is enabled (which might be worth doing
regardless). This is what PR #1659 does. However, I think the approach
in this branch is more correct; the feature flags that require standard
library support should automatically enable it. This provides a better
experience for most users, who are simply adding `default-features = false`
in order to avoid enabling un-needed features, not to support
`no_std`, and would like enabling a feature flag to *actually* enable
that feature.
`no_std` users will know not to enable `registry`, `fmt`, and
`env-filter` because the documentation notes that those features require
the standard library.
Backports #1648 from `master`.
Depends on #1649
## Motivation
Presently, the `tracing-subscriber` crate requires the Rust standard
library and doesn't build with `#![no_std]` targets. For the most part,
this is fine, as much of `tracing-subscriber` inherently depends on
`std` APIs.
However, `tracing-subscriber` also contains some key abstractions that
are necessary for interoperability: the `Layer` and `LookupSpan`
traits. Since these traits are in `tracing-subscriber`, `no-std` users
cannot currently access them.
Some of the other utilities in this crate, such as the field visitor
combinators, may also be useful for `#![no_std]` projects.
## Solution
This branch adds "std" and "alloc" feature flags to
`tracing-subscriber`, for conditionally enabling `libstd` and
`liballoc`, respectively. The `registry`, `fmt`, `EnvFilter`, and
`reload` APIs all require libstd, and cannot be implemented without it,
but the core `Layer` and `LookupSpan` traits are now available with
`#![no_std]`.
Fixes#999
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Implementing `Layer` for `Arc`s, which are immutable, breaks the
ability to implement `Layer::on_layer` with a mutable reference.
This is necessary for per-layer filtering. See
https://github.com/tokio-rs/tracing/pull/1576#discussion_r711609810 for
details. Therefore, the `Layer` impls for `Arc`s should not be used.
In 0.3, we have the opportunity to remove these APIs. Therefore, this PR
removes them.
This changes `tracing-subscriber` so that the `env-filter`, `json`,
and `chrono` features are not enabled by default, and instead require
users to opt in. This should significantly reduce the default
dependency footprint.
Of course, this is a breaking change, and therefore will be part of
`tracing-subscriber` v0.3.
Fixes#1258
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Currently, `tracing-subscriber` supports the `chrono` crate for
timestamp formatting, via a default-on feature flag. When this code was
initially added to `tracing-subscriber`, the `time` crate did not have
support for the timestamp formatting options we needed.
Unfortunately, the `chrono` crate's maintainance status is now in
question (see #1598). Furthermore, `chrono` depends on version 0.1 of
the `time` crate, which contains a security vulnerability
(https://rustsec.org/advisories/RUSTSEC-2020-0071.html). This
vulnerability is fixed in more recent releases of `time`, but `chrono`
still uses v0.1.
## Solution
Fortunately, the `time` crate now has its own timestamp formatting
support.
This branch replaces the `ChronoLocal` and `ChronoUtc` timestamp
formatters with new `LocalTime` and `UtcTime` formatters. These
formatters use the `time` crate's formatting APIs rather than
`chrono`'s. This removes the vulnerable dependency on `time` 0.1
Additionally, the new `time` APIs are feature flagged as an _opt-in_
feature, rather than as an _opt-out_ feature. This should make it easier
to avoid accidentally depending on the `time` crate when more
sophisticated timestamp formatting is _not_ required.
In a follow-up branch, we could also add support for `humantime` as an
option for timestamp formatting.
Naturally, since this removes existing APIs, this is a breaking change,
and will thus require publishing `tracing-subscriber` 0.3. We'll want to
do some other breaking changes as well.
Fixes#1598.
Previously, the documentation explained that an `EnvFilter` consisted
of multiple directives, but didn't describe how to actually put
these directives together. (My first guess was "with spaces", but
that was wrong.)
This patch changes the documentation to say that directives are
comma-separated, and gives an example of a comma-separated filter.
I'm quite confident that no elsewhere using the
`tracing_subscriber::CurrentSpan` in the present codebases. Do we need
to keep it for the future version? Otherwise, I think we can remove it
and the related `thread.rs`.
Feel free to point me out if I'm wrong. :)
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
This backports PR #781 from `master`.
## Motivation
Currently, the `tracing-subscriber` crate has the `MakeWriter` trait for
customizing the io writer used by `fmt`. This trait is necessary (rather
than simply using a `Write` instance) because the default implementation
performs the IO on the thread where an event was recorded, meaning that
a separate writer needs to be acquired by each thread (either by calling
a function like `io::stdout`, by locking a shared `Write` instance,
etc).
Right now there is a blanket impl for `Fn() -> T where T: Write`. This
works fine with functions like `io::stdout`. However, the _other_ common
case for this trait is locking a shared writer.
Therefore, it makes sense to see an implementation like this:
``` rust
impl<'a, W: io::Write> MakeWriter for Mutex<W>
where
W: io::Write,
{
type Writer = MutexWriter<'a, W>;
fn make_writer(&self) -> Self::Writer {
MutexWriter(self.lock().unwrap())
}
}
pub struct MutexWriter<'a, W>(MutexGuard<'a, W>);
impl<W: io::Write> io::Write for MutexWriter<'_, W> {
// write to the shared writer in the `MutexGuard`...
}
```
Unfortunately, it's impossible to write this. Since `MakeWriter` always
takes an `&self` parameter and returns `Self::Writer`, the generic
parameter is unbounded:
```
Checking tracing-subscriber v0.2.4 (/home/eliza/code/tracing/tracing-subscriber)
error[E0207]: the lifetime parameter `'a` is not constrained by the impl trait, self type, or predicates
--> tracing-subscriber/src/fmt/writer.rs:61:6
|
61 | impl<'a, W: io::Write> MakeWriter for Mutex<W>
| ^^ unconstrained lifetime parameter
error: aborting due to previous error
```
This essentially precludes any `MakeWriter` impl where the writer is
borrowed from the type implementing `MakeWriter`. This is a significant
blow to the usefulness of the trait. For example, it prevented the use
of `MakeWriter` in `tracing-flame` as suggested in
https://github.com/tokio-rs/tracing/pull/631#discussion_r391138233.
## Proposal
This PR changes `MakeWriter` to be generic over a lifetime `'a`:
```rust
pub trait MakeWriter<'a> {
type Writer: io::Write;
fn make_writer(&'a self) -> Self::Writer;
}
```
The `self` parameter is now borrowed for the `&'a` lifetime, so it is
okay to return a writer borrowed from `self`, such as in the `Mutex`
case.
I've also added an impl of `MakeWriter` for `Mutex<T> where T: Writer`.
Unfortunately, this is a breaking change and will need to wait until we
release `tracing-subscriber` 0.3.
Fixes#675.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
I'm trying to implement a `Layer` that has something very similar to
tracing-error's
[`WithContext`](66cd79f72a/tracing-error/src/layer.rs (L32))
to support erasing types and getting access to the current span's state.
To do that, I implement `Layer::downcast_raw` in [the same way that
tracing-error
does](66cd79f72a/tracing-error/src/layer.rs (L55-L63)).
This works great when the layer is not filtered. However, once I filter
the layer
```rust
let filter = tracing_subscriber::filter::Targets::new().with_default(tracing::Level::INFO);
let layer = MyLayer::new();
tracing_subscriber::registry().with(layer.with_filter(filter)).init();
```
I'm not able to get a `WithContext` instance anymore, because `Filtered`
[handles `downcast_raw`, and doesn't forward
it](66cd79f72a/tracing-subscriber/src/filter/layer_filters.rs (L379-L391))
to `MyLayer::downcast_raw`.
## Solution
If `Filtered::downcast_raw` does not know how to handle the given type,
forward it to the wrapped layer's `Layer::downcast_raw` implementation.
Fixes#1618
This PR contains two changes:
1. An example as how to use `Option<Layer>` to toggle layers at runtime, and...
2. A fix to an example writing to disk when doc tests are run.
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
# 0.2.24 (September 19, 2021)
This release contains a number of bug fixes, including a fix for
`tracing-subscriber` failing to compile on the minimum supported Rust
version of 1.42.0. It also adds `IntoIterator` implementations for the
`Targets` type.
### Fixed
- Fixed compilation on Rust 1.42.0 ([#1580], [#1581])
- **registry**: Ensure per-layer filter `enabled` state is cleared when
a global filter short-circuits filter evaluation ([#1575])
- **layer**: Fixed `Layer::on_layer` not being called for `Box`ed
`Layer`s, which broke per-layer filtering ([#1576])
### Added
- **filter**: Added `Targets::iter`, returning an iterator over the set
of target-level pairs enabled by a `Targets` filter ([#1574])
- **filter**: Added `IntoIterator` implementations for `Targets` and
`&Targets` ([#1574])
Thanks to new contributor @connec for contributing to this release!
[#1580]: https://github.com/tokio-rs/tracing/pull/1580
[#1581]: https://github.com/tokio-rs/tracing/pull/1581
[#1575]: https://github.com/tokio-rs/tracing/pull/1575
[#1576]: https://github.com/tokio-rs/tracing/pull/1576
[#1574]: https://github.com/tokio-rs/tracing/pull/1574
This is essentially the same change as #1569, but for `enabled` states
rather than `register_callsite`. When a global filter returns `false`
from `enabled`, ensure that the per-layer filter `FilterMap` and debug
counters are cleared, so that they are empty on the next `enabled` call.
See #1563
Due to the CI issue fixed by PR #1580, we accidentally merged
`tracing-subscriber` changes that don't compile on our MSRV, Rust
1.42.0. This includes:
- use of associated consts on primitive types (`u64::MAX` rather than
`std::u64::MAX`)
- use of `#[cfg(...)]` on `if` expressions
- use of the `str::strip_suffix` method, which wasn't stable on 1.42.0
This commit fixes these issues. These changes are mostly very simple and
mechanical. The only significant change is changing the
`StaticDirective` parsing code to not use `strip_suffix`, which changes
the implementation around a bit...but it's still pretty straightforward.
This should fix 1.42.0 compatibility.
There's currently no way to get the registered directives from a
`Targets` instance, which is a barrier to some use-cases such as
combining user-supplied directives with application-supplied defaults.
This commit adds a `Targets::targets` method, which returns an iterator
of `(&str, LevelFilter)` pairs, one for each directive (excluding the
default level, whose `target` is `None`).
Items are yielded as per the underlying `DirectiveSet::directives`,
which would yield itms in specifity order (as constructed by
`DirectiveSet::add`). However, the order has been left explicitly
undefined in the documentation for `Targets::targets` on the basis that
this is an implementation detail that may change.
## Motivation
As hinted in the commit message, my use-case is to have an
application-supplied 'base' `Targets` filter, which can then be
extended/overridden by a user-supplied filter parsed from `RUST_LOG`
(e.g.).
Sadly, there's currently no way of getting the directives out of a
`Targets` instance, nor to re-serialize to the string representation.
Thus, the only way to implement the above would be to manually parse the
user-supplied filters in the application, which is shame since it
duplicates the implementation here and would be prone to drift/subtle
incompatibilities.
## Solution
The proposed solution is to add methods to `Targets` to retrieve the
underlying directives.
```rust
// application-defined defaults, for a useful level of information
let mut filter = Targets::new()
.with_target("my_crate", LevelFilter::INFO)
.with_target("important_dependency", LevelFilter::INFO);
if let Ok(overrides) = std::env::var("RUST_LOG") {
let overrides: Targets = overrides.parse().unwrap();
// merge user-supplied overrides, which can enable additional tracing
// or disable default tracing
filter.extend(overrides.iter());
// ^^^^^^^^^ this is new
}
```
For this PR, I've gone for `fn iter(&self) -> Iter`, e.g. a method
`targets` that returns an iterator over the pairs of `(&str,
LevelFilter)` in the underlying directives. The iterator excludes any
default level(s) where `target` would be `None`. This matches nicely
with the `FromIterator` and `Extend` impls, which use `(impl
Into<String>, impl Into<LevelFilter>)`.
In addition, I've added `IntoIterator` implementations for `&Targets`
and for `Targets`. The by-value `IntoIterator` implementation returns an
`IntoIter` type that moves the targets as `String`s, rather than as
borrowed `&str`s. This makes `extend` more efficient when moving a
second `Targets` by value.
The `Layer::on_layer` method on `Layer` was added in PR #1523. PR #1536,
which added `Layer` implementations to `Box<dyn Layer<...> + ...>` and
`Arc<dyn Layer<...> + ...>`, merged prior to #1523. However, when
merging #1523, I didn't think to update the `Layer` impl for `Box`ed and
`Arc`ed layers to forward `on_layer` to the inner `Layer`. This means
that when a `Layer` is wrapped in an `Arc` or a `Box`, the `on_layer`
method never gets called.
When per-layer filters are in use, the `on_layer` method is necessary to
ensure the filter is registered with the inner subscriber and has a
valid ID. This bug means that when per-layer filters are wrapped in a
`Box` or `Arc`, they won't be registered, and per-layer filtering
breaks.
This PR fixes the bug by adding `on_layer` implementations to the
`Layer` impls for `Arc`ed and `Box`ed layers. I also added some tests
--- thanks to @Noah-Kennedy for the original repro that these were based
on (https://github.com/tokio-rs/tracing/issues/1563#issuecomment-922014777).
I also added a nicer debug assertion to `Filtered` for cases where
`Layer` impls don't call `on_layer`, so that this fails less confusingly
in the future.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
This branch removes the HTML `<div>` tags from the warning and note
blocks in the documentation, because they're screwing up RustDoc
rendering on docs.rs. We had done this previously on the `master`
branch, but apparently it never made it to `v0.1.x`.
This should fix#1571 once a new `tracing` version is published.
Currently, when evaluating `register_callsite` for a stack containing
per-layer filters, the intermediate `Interest` from combining the per
layer filters' `Interest`s is stored in the thread-local `FilterState`.
When all per-layer filters have been evaluated, we reach the `Registry`,
which clears the `FilterState` and bubbles the `Interest` back up.
However, when a _global_ filter in the stack returns `Interest::never`,
we short-circuit, and don't reach the `Registry`. This means the
`Interest` state is not cleared.
This branch adds code in `Layered` to ensure the per-layer filter state
is cleared when a global filter short circuits `Interest` evaluation.
This fixes#1563.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
The `DirectiveSet` type used in `EnvFilter` and `Targets` uses
`SmallVec` to store the filtering directives when the `SmallVec` feature
is enabled. This is intended to improve the performance of iterating
over small sets of directives, by avoiding a heap pointer dereference.
PR #1550 changed the directives themselves to also use `SmallVec` for
storing _field_ filters. This was intended to make the same optimization
for field filters. However, it had unintended consequences: an empty
`SmallVec` is an array of `T` items (plus metadata), while an empty
`Vec` is just a couple of words. Since _most_ filters don't have field
filters, this meant that we were suddenly using a lot more space to
store...nothing. This made `EnvFilter`s _much_ larger, causing problems
for some users (see #1567).
## Solution
This branch undoes the change to `SmallVec` for field name/value
filters. This takes the size of an `EnvFilter` from 5420 bytes back down
to 1272 bytes.
I also added some tests that just print the size of various `EnvFilter` and
`Targets` values. These don't make any assertions, but can be run for
development purposes when making changes to these types.
Fixes#1567
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
# 0.2.22 (September 13, 2021)
This fixes a regression where the `filter::ParseError` type was
accidentally renamed.
### Fixed
- **filter**: Fix `filter::ParseError` accidentally being renamed to
`filter::DirectiveParseError` ([#1558])
[#1558]: https://github.com/tokio-rs/tracing/pull/1558
* small doc cleanups
* subscriber: Use NoSubscriber in Layer tests
No need for another no-op subscriber.
* subscriber: add some compile-time tests for dyn Layer
Make sure that `Box<dyn Layer<S>>` and `Arc<dyn Layer<S>>` meet all the
constraints to be actually usable.
I accidentally renamed this type in PR #1550 after getting confused by a
renaming import that I thought was publicly re-exported, but it wasn't
actually..sorry about that!
This commit renames (or...un-renames?) the type. Whoopsie!
Fixes#1557
# 0.2.21 (September 12, 2021)
This release introduces the [`Filter`] trait, a new API for [per-layer
filtering][plf]. This allows controlling which spans and events are
recorded by various layers individually, rather than globally.
In addition, it adds a new [`Targets`] filter, which provides a
lighter-weight version of the filtering provided by [`EnvFilter`], as
well as other smaller API improvements and fixes.
### Deprecated
- **registry**: `SpanRef::parent_id`, which cannot properly support
per-layer filtering. Use `.parent().map(SpanRef::id)` instead.
([#1523])
### Fixed
- **layer** `Context` methods that are provided when the `Subscriber`
implements `LookupSpan` no longer require the "registry" feature flag
([#1525])
- **layer** `fmt::Debug` implementation for `Layered` no longer requires
the `S` type parameter to implement `Debug` ([#1528])
### Added
- **registry**: `Filter` trait, `Filtered` type, `Layer::with_filter`
method, and other APIs for per-layer filtering ([#1523])
- **filter**: `FilterFn` and `DynFilterFn` types that implement global
(`Layer`) and per-layer (`Filter`) filtering for closures and function
pointers ([#1523])
- **filter**: `Targets` filter, which implements a lighter-weight form
of `EnvFilter`-like filtering ([#1550])
- **env-filter**: Added support for filtering on floating-point values
([#1507])
- **layer**: `Layer::on_layer` callback, called when layering the
`Layer` onto a `Subscriber` ([#1523])
- **layer**: `Layer` implementations for `Box<L>` and `Arc<L>` where `L:
Layer` ([#1536])
- **layer**: `Layer` implementations for `Box<dyn Layer>` and
`Arc<dyn Layer>` ([#1536])
- A number of small documentation fixes and improvements ([#1553],
[#1544], [#1539], [#1524])
Special thanks to new contributors @jsgf and @maxburke for contributing
to this release!
[`Filter`]: https://docs.rs/tracing-subscriber/0.2.21/tracing_subscriber/layer/trait.Filter.html
[plf]: https://docs.rs/tracing-subscriber/0.2.21/tracing_subscriber/layer/index.html#per-layer-filtering
[`Targets`]: https://docs.rs/tracing-subscriber/0.2.21/tracing_subscriber/filter/struct.Targets.html
[`EnvFilter`]: https://docs.rs/tracing-subscriber/0.2.21/tracing_subscriber/filter/struct.EnvFilter.html
[#1507]: #1507
[#1523]: #1523
[#1524]: #1524
[#1525]: #1525
[#1528]: #1528
[#1539]: #1539
[#1544]: #1544
[#1550]: #1550
[#1553]: #1553
This branch adds some new documentation in a few places that references
other sections in the docs. This should help make some of the crate's
APIs easier to find. In particular, the crate's top level docs now
summarize the `Layer` and `Filter` traits, and link to their individual
docs sections. Additionally, the per-layer filtering section in the
`Layer` module docs now references some of the concrete `Filter`
implementations in the `filter` module.
This branch adds a new `Targets` filter to `tracing_subscriber`. The
`Targets` filter is very similar to `EnvFilter`, but it _only_ consists
of filtering directives consisting of a target and level. Because it
doesn't support filtering on field names, span contexts, or field
values, the implementation is *much* simpler, and it doesn't require the
`env_filter` feature flag. Also, `Targets` can easily implement the
`Filter` trait for per-layer filtering, while adding a `Filter`
implementation for `EnvFilter` will require additional effort.
Because the `Targets` filter doesn't allow specifiyng span or
field-value filters, the syntax for parsing one from a string is
significantly simpler than `EnvFilter`'s. Therefore, it can have a very
simple handwritten parser implementation that doesn't require the
`regex` crate. This should be useful for users who are concerned about
the number of dependencies required by `EnvFilter`.
The new implementation is quite small, as it mostly uses the same code
as the static filter subset of `EnvFilter`. This code was factored out
into a shared module for use in both `EnvFilter` and `Targets`. The code
required for _dynamic_ filtering with `EnvFilter` (i.e. on fields and
spans) is still in the `filter::env` module and is only enabled by the
`env-filter` feature flag.
I'm open to renaming the new type; I thought `filter::Targets` seemed
good, but would also be willing to go with `TargetFilter` or something.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Currently, some types in the `filter` module that require the
"env-filter" feature flag don't properly show in the docs that they
require this feature. This is because the `doc(cfg(feature = "env-filter"))`
attribute is only placed on the _re-export_ from the
`env` module (and on the `EnvFilter` type), not on the individual
types that are re-exported. However, placing a `doc(cfg(...))` attribute
on a re-export doesn't actually do anything.
## Solution
This commit fixes that.
## Motivation
Currently, the `FilterFn` and `DynFilterFn` filters are only usable as
`Filter`s for per-layer filtering. However, there's no real reason they
can't _also_ have `Layer` implementations, making them usable as global
filters as well.
## Solution
This branch adds `Layer` implementations to `FilterFn` and
`DynFilterFn`, and moves them into their own file. It also changes the
feature flagging so that the "registry" feature is only required for the
`Filter` implementations --- the types themselves don't involve any
feature-flagged code, only the `Filter` trait.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Currently, filtering with `Layer`s is always _global_. If a `Layer`
performs filtering, it will disable a span or event for _all_ layers
that compose the current subscriber. In some cases, however, it is often
desirable for individual layers to see different spans or events than
the rest of the `Layer` stack.
Issues in other projects, such as tokio-rs/console#64 and
tokio-rs/console#76, linkerd/linkerd2-proxy#601,
influxdata/influxdb_iox#1012 and influxdata/influxdb_iox#1681,
jackwaudby/spaghetti#86, etc; as well as `tracing` feature requests like
#302, #597, and #1348, all indicate that there is significant demand for
the ability to add filters to individual `Layer`s.
Unfortunately, doing this nicely is somewhat complicated. Although a
naive implementation that simply skips events/spans in `Layer::on_event`
and `Layer::new_span` based on some filter is relatively simple, this
wouldn't really be an ideal solution, for a number of reasons. A proper
per-layer filtering implementation would satisfy the following
_desiderata_:
* If a per-layer filter disables a span, it shouldn't be present _for
the layer that filter is attached to_ when iterating over span
contexts (such as `Context::event_scope`, `SpanRef::scope`, etc), or
when looking up a span's parents.
* When _all_ per-layer filters disable a span or event, it should be
completely disabled, rather than simply skipped by those particular
layers. This means that per-layer filters should participate in
`enabled`, as well as being able to skip spans and events in
`new_span` and `on_event`.
* Per-layer filters shouldn't interfere with non-filtered `Layer`s.
If a subscriber contains layers without any filters, as well as
layers with per-layer filters, the non-filtered `Layer`s should
behave exactly as they would without any per-layer filters present.
* Similarly, per-layer filters shouldn't interfere with global
filtering. If a `Layer` in a stack is performing global filtering
(e.g. the current filtering behavior), per-layer filters should also
be effected by the global filter.
* Per-layer filters _should_ be able to participate in `Interest`
caching, _but only when doing so doesn't interfere with
non-per-layer-filtered layers_.
* Per-layer filters should be _tree-shaped_. If a `Subscriber` consists
of multiple layers that have been `Layered` together to form new
`Layer`s, and some of the `Layered` layers have per-layer filters,
those per-layer filters should effect all layers in that subtree.
Similarly, if `Layer`s in a per-layer filtered subtree have their
_own_ per-layer filters, those layers should be effected by the union
of their own filters and any per-layer filters that wrap them at
higher levels in the tree.
Meeting all these requirements means that implementing per-layer
filtering correctly is somewhat more complex than simply skipping
events and spans in a `Layer`'s `on_event` and `new_span` callbacks.
## Solution
This branch has a basic working implementation of per-layer filtering
for `tracing-subscriber` v0.2. It should be possible to add this in a
point release without a breaking change --- in particular, the new
per-layer filtering feature _doesn't_ interfere with the global
filtering behavior that layers currently have when not using the new
per-layer filtering APIs, so existing configurations should all behave
exactly as they do now.
### Implementation
The new per-layer filtering API consists of a `Filter` trait that
defines a filtering strategy and a `Filtered` type that combines a
`Filter` with a `Layer`. When `enabled` is called on a `Filtered` layer,
it checks the metadata against the `Filter` and stores whether that
filter enabled the span/event in a thread-local cell. If every per-layer
filter disables a span/event, and there are no non-per-layer-filtered
layers in use, the `Registry`'s `enabled` will return `false`.
Otherwise, when the span/event is recorded, each `Filtered` layer's
`on_event` and `new_span` method checks with the `Registry` to see
whether _it_ enabled or disabled that span/event, and skips passing it
to the wrapped layer if it was disabled. When a span is recorded, the
`Registry` which filters disabled it, so that they can skip it when
looking up spans in other callbacks.
A new `on_layer` method was added to the `Layer` trait, which allows
running a callback when adding a `Layer` to a `Subscriber`. This allows
mutating both the `Layer` and the `Subscriber`, since they are both
passed by value when layering a `Layer` onto a `Subscriber`, and a new
`register_filter` method was added to `LookupSpan`, which returns a
`FilterId` type. When a `Filtered` layer is added to a subscriber that
implements `LookupSpan`, it calls `register_filter` in its `on_layer`
hook to generate a unique ID for its filter. `FilterId` can be used to
look up whether a span or event was enabled by the `Filtered` layer.
Internally, the filter results are stored in a simple bitset to reduce
the performance overhead of adding per-layer filtering as much as
possible. The `FilterId` type is actually a bitmask that's used to set
the bit corresponding to a `Filtered` layer in that bitmap when it
disables a span or event, and check whether the bit is set when querying
if a span or event was disabled. Setting a bit in the bitset and testing
whether its set is extremely efficient --- it's just a couple of bitwise
ops. Additionally, the "filter map" that tracks which filters disabled a
span or event is just a single `u64` --- we don't need to allocate a
costly `HashSet` or similar every time we want to filter an event. This
*does* mean that we are limited to 64 unique filters per
`Registry`...however, I would be _very_ surprised if anyone _ever_
builds a subscriber with 64 layers, let alone 64 separate per-layer
filters.
Additionally, the `Context` and `SpanRef` types now also potentially
carry `FilterId`s, so that they can filter span lookups and span
traversals for `Filtered` layers. When `Filtered` layers are nested
using the `Layered`, the `Context` _combines_ the filter ID of the inner
and outer layers so that spans can be skipped if they were disabled by
either.
Finally, an implementation of the `Filter` trait was added for
`LevelFilter`, and new `FilterFn` and `DynFilterFn` structs were added
to produce a `Filter` implementation from a function pointer or closure.
### Notes
Some other miscellaneous factors to consider when reviewing this change
include:
* I did a bit of unrelated refactoring in the `layer` module. Since we
were adding more code to the `Layer` module (the `Filter` trait), the
`layer.rs` file got significantly longer and was becoming somewhat
hard to navigate. Therefore, I split out the `Context` and `Layered`
types into their own files. Most of the code in those files is the
same as it was before, although some of it was changed in order to
support per-layer filtering. Apologies in advance to reviewers for
this...
* In order to test this change, I added some new test-support code in
`tracing-subscriber`. The `tracing` test support code provides a
`Subscriber` implementation that can be configured to expect a
particular set of spans and events, and assert that they occurred as
expected. In order to test per-layer filtering, I added a new `Layer`
implementation that does the same thing, but implements `Layer`
rather than `Subscriber`. This allows testing per-layer filter
behavior in the same way we test global filter behavior. Reviewers
should feel free to just skim the test the test support changes, or
skip reviewing them entirely.
## Future Work
There is a bunch of additional stuff we can do once this change lands.
In order to limit the size of this PR, I didn't make these changes yet,
but once this merges, we will want to consider some important follow
up changes:
* [ ] Currently, _only_ `tracing-subscriber`'s `Registry` is capable of
serving as a root subscriber for per-layer filters. This is in
order to avoid stabilizing a lot of the per-layer filtering
design as public API without taking the time to ensure there are
no serious issues. In the future, we will want to add new APIs to
allow users to implement their own root `Subscriber`s that can
host layers with per-layer filtering.
* [ ] The `EnvFilter` type doesn't implement the `Filter` trait and thus
cannot currently be used as a per-layer filter. We should add an
implementation of `Filter` for `EnvFilter`.
* [ ] The new `FilterFn` and `DynFilterFn` types could conceivably _also_
be used as global filters. We could consider adding `Layer`
implementations to allow them to be used for global filtering.
* [ ] We could consider adding new `Filter` implementations for performing
common filtering predicates. For example, "only enable spans/events
with a particular target or a set of targets", "only enable spans",
"only enable events", etc. Then, we could add a combinator API to
combine these predicates to build new filters.
* [ ] It would be nice if the `Interest` system could be used to cache
the result of multiple individual per-layer filter evaluations, so
that we don't need to always use `enabled` when there are several
per-layer filters that disagree on whether or not they want a
given span/event. There are a lot of unused bits in `Interest`
that could potentially be used for this purpose. However, this
would require new API changes in `tracing-core`, and is therefore
out of scope for this PR.
Closes#302Closes#597Closes#1348
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
I noticed a handful of places where `v0.1.x` refers to `Subscriber`s as
"collectors". This probably happened because we backported some commits
from master and forgot to change every instance of "collector" back to
"subscriber".
This commit fixes that. Whoops.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
## Motivation
Tracing is a really useful framework but a lack of floating point value
support for the core visitors means these get coerced unnecessarily to
strings.
## Solution
This change adds support for floating point (`f64`) visitors.
