175 Commits

Author SHA1 Message Date
René Kijewski
35a77e3c25 Fix winnow deprecation warning 2025-01-12 19:43:52 +01:00
René Kijewski
ff25aaab2a parser: fix excessive filter blocks
A `{% filter %}` block could have an unbound number of filters chained
one after another. This chain would wind up as a nested structure on the
heap. When dropped, this would cause a stackoverflow.

Resolves <https://issues.oss-fuzz.com/issues/386547457>.
2025-01-04 11:23:54 +01:00
René Kijewski
4b8bd45844 parser: use drop guards to track nesting level
Not for all operations the nesting level was incremented when needed
and/or the un-incremented nesting level was used in subfunction calls.
Binary operators such as `*` did not properly increment the nesting
level.

This PR changes `Level` in such a way that it can be used to keep track
of the nesting level when used in a loop. It is now impossible to
accidentally refer to an old nesting level value.

Resolves <https://issues.oss-fuzz.com/issues/385256115>.
2024-12-27 04:31:24 +01:00
Guillaume Gomez
5cc7cb2121 Add full support for if let chain feature 2024-12-25 21:18:57 +01:00
René Kijewski
18f1d9b343 parser: deny rust 2024 keyword gen as target name 2024-12-21 23:50:13 +01:00
René Kijewski
c80e78e8b5 parser: deny all rust keywords as target names 2024-12-21 23:49:48 +01:00
Guillaume Gomez
2a9d240ac4 Improve reserved variable name error message 2024-12-21 23:06:46 +01:00
René Kijewski
0cc0189927 Add missing softlinks 2024-12-21 19:55:15 +01:00
René Kijewski
e800b7ca90 derive: reserve variable prefix __rinja and free name writer 2024-12-21 19:42:03 +01:00
René Kijewski
bedc31797c Make #![no_std] compatible 2024-12-21 18:53:44 +01:00
Ed Page
64b4893f93 refactor(parser): Upgrade to Winnow 0.6 2024-12-20 10:36:24 -06:00
Ed Page
47a7334321
Upgrade from Winnow 0.4 to 0.5 (#214)
* refactor(parser): Consistently use 'i' in skip_till

* refactor(parser): Clarify second input in skip_till

* refactor(parser): Consistently use 'i' in Comment::parse::content

* refactor(parser): Clarify second input in Comment::parse::content

* refactor(parser): Rename ParseResult to InputParseResult

This leaves room for a new `ParseResult` that doesn't have the input as
we transition to the Winnow 0.5 API.

* refactor(parser): Temporarily switch from parse_next to parse_peek

In 0.4, `parse_next` and `parse_peek` are the same.
In 0.5, `parse_next`s signature will change.
By switching to `parse_peek` now, we can make it so we gradually adopt
the new `parse_next` signature.

* refactor(parser): Temporarilly add unpeeks

In 0.4, `unpeek` is a no-op.
In 0.5, the `FnMut` signature impled for `Parser` changes and `unpeek`
will take the old signature and adapt it to work.
This allows us to more gradually migrate to the new `FnMut` signature.

Yes, the name `unpeek` isn't ideal but its only transitional.

* refactor(parser): Upgrade to Winnow 0.5

Note: `tests::fuzzed_target_recursion` fails on debug builds because of
the extra overhead from the deprecated APIs adapting to their
non-deprecated forms

* refactor(parser): Update signature for 'identifier'

* refactor(parser): Update signature for 'bool_lit'

* refactor(parser): Update signature for 'num_lit_suffix'

* refactor(parser): Update body for 'num_lit::int_with_base'

* refactor(parser): Update signature for 'num_lit::float'

* refactor(parser): Update signature for 'num_lit'

* refactor(parser): Update signature for 'str_lit_without_prefix'

* refactor(parser): Update signature for 'str_lit'

* refactor(parser): Update signature for 'char_lit'

* refactor(parser): Update signature for 'Char::parse'

* refactor(parser): Update signature for 'path_or_identifier'

* refactor(parser): Update signature for 'filter'

* refactor(parser): Update signature for 'State::tag_block_start'

* refactor(parser): Update signature for 'State::tag_block_end'

* refactor(parser): Update signature for 'State::tag_comment_start'

* refactor(parser): Update signature for 'State::tag_comment_end'

* refactor(parser): Update signature for 'State::tag_expr_start'

* refactor(parser): Update signature for 'State::tag_expr_end'

* refactor(parser): Update signature for 'State::nest'

* refactor(parser): Update signature for 'Level::nest'

I had considered making the input a `&mut &str` but didn't really see
the benefit, especially at this stage.
That can always be changed later.

* refactor(parser): Update signature for 'skip_ws0'

* refactor(parser): Update signature for 'skip_ws1'

* refactor(parser): Update signature for 'separated_digits' closure

* refactor(parser): Update signature for 'skip_till' closure

* refactor(parser): Update signature for 'Target::lit'

* refactor(parser): Update signature for 'Target::unnamed'

* refactor(parser): Update signature for 'Target::named'

* refactor(parser): Update signature for 'Target::rest'

* refactor(parser): Update signature for 'collect_targets'

* refactor(parser): Update signature for 'Target::parse'

* refactor(parser): Update signature for 'Target::parse_one'

* refactor(parser): Update signature for 'Suffix::try'

* refactor(parser): Update signature for 'Suffix::call'

* refactor(parser): Update signature for 'Suffix::index'

* refactor(parser): Update signature for 'Suffix::attr'

* refactor(parser): Update signature for 'Suffix::macro'

* refactor(parser): Update signature for 'token_bitand'

* refactor(parser): Update signature for 'token_xor'

* refactor(parser): Update signature for 'Expr::char'

* refactor(parser): Update signature for 'Expr::num'

* refactor(parser): Update signature for 'Expr::str'

* refactor(parser): Update signature for 'Expr::path_var_bool'

* refactor(parser): Update signature for 'Expr::array'

* refactor(parser): Update signature for 'Expr::group'

* refactor(parser): Update signature for 'Expr::single'

* refactor(parser): Update signature for 'Expr::prefix'

* refactor(parser): Update signature for 'Expr::filtered'

* refactor(parser): Update signature for 'Expr::is_as'

* refactor(parser): Update signature for 'Expr::concat'

* refactor(parser): Update signature for 'expr_prec_layer'

* refactor(parser): Update signature for 'Suffix::parse'

* refactor(parser): Update signature for 'Expr::parse'

* refactor(parser): Update signature for 'Expr::named_argument'

* refactor(parser): Update signature for 'Expr::arguments'

* refactor(parser): Update signature for 'Expr::arguments' closure

* refactor(parser): Update signature for 'Comment::parse::tag

* refactor(parser): Update signature for 'Comment::parse::content

* refactor(parser): Update signature for 'Comment::parse'

* refactor(parser): Update signature for 'Extends::parse'

* refactor(parser): Update signature for 'Include::parse'

* refactor(parser): Update signature for 'Import::parse'

* refactor(parser): Update signature for 'Whitespace::parse'

* refactor(parser): Update signature for 'Lit::parse'

* refactor(parser): Update signature for 'Node::expr'

* refactor(parser): Update signature for 'Node::many'

* refactor(parser): Update signature for 'Node::parse'

* refactor(parser): Update signature for 'Node::parse_template'

* refactor(parser): Update signature for 'cut_node' closure

* refactor(parser): Avoid 'parse_peek' in 'cut_node'

* refactor(parser): Update signature for 'unexpected_raw_tag'

* refactor(parser): Update signature for 'unexpected_tag'

* refactor(parser): Update signature for 'check_end_name'

* refactor(parser): Update signature for 'CondTest::parse_cond'

* refactor(parser): Update signature for 'check_block_start'

* refactor(parser): Update signature for 'Loop::parse::content'

* refactor(parser): Update signature for 'Macro::parse::parameters'

* refactor(parser): Update signature for 'Loop::parse::else_block'

* refactor(parser): Update signature for 'Loop::parse::body_and_end'

* refactor(parser): Reuse built-in empty parser

* refactor(parser): Update signature for remaining '*::parse'

* refactor(parser): Resolve remaining deprecations
2024-12-20 17:25:20 +01:00
Guillaume Gomez
43a0efda7c
Merge pull request #276 from Kijewski/pr-one-CompileError
derive: less `format!()`, shorter context on error
2024-12-13 12:02:49 +01:00
René Kijewski
b49e42f0ea Use cargo reference manifest section order
In [#131] I added a check that ensures that the section in `Cargo.toml`
(and there values) are in a specific order. I invented the order quite
randomly, in the order which I thought might be useful.

This PR changes the order to be the same as in Cargo's reference about
[The Manifest Format]. Also some missing softlinks were added.

[#131]: <https://github.com/rinja-rs/rinja/pull/131>
[The Manifest Format]: <https://doc.rust-lang.org/cargo/reference/manifest.html>
2024-12-13 10:00:48 +01:00
René Kijewski
657b70d8bb Remove stray commas 2024-12-13 02:54:43 +01:00
René Kijewski
430b99bdac Fix clippy warnings in rust nightly 1.85 2024-12-11 19:50:29 +01:00
Ed Page
e31f4e210e
refactor(parser): Be explicit about input state (#275)
* refactor(parser): In 'filter', name a checkpoint

* refactor(parser): In 'Target::named', name a checkpoint

* refactor(parser): In 'Target::named', remove a checkpoint

* refactor(parser): In 'Target::rest', name a checkpoint

* refactor(parser): In 'Target::parse_one', name a checkpoint

* refactor(parser): In 'Target::parse', align the input

* refactor(parser): In 'Expr::is_as', name a checkpoint

* refactor(parser): In 'Expr::concat', name a checkpoint

* refactor(parser): In 'Suffix::parse', name a checkpoint

* refactor(parser): In 'Node::break', name a checkpoint

* refactor(parser): In 'Node::continue', name a checkpoint

* refactor(parser): In 'Macro::parse', name a checkpoint

* refactor(parser): In 'When::else', name a checkpoint
2024-11-25 23:56:11 +01:00
René Kijewski
47c3f6cc65 parser: shrink the size of WithSpan to one register 2024-11-24 22:30:47 +01:00
René Kijewski
8055e37411 Unify Whitespace and WhitespaceHandling 2024-11-24 01:08:51 +01:00
René Kijewski
f7917d57c1 parser: remove obsolete parser fuzzing
The fuzzing we inherited from askama only tested the parser. In rinja we
parse multiple aspects of the project in a unified crate: `fuzzing`.
Everything the old fuzzer did is included in the new fuzzer, and more.
2024-11-20 23:55:51 +01:00
Guillaume Gomez
8dbbb9ae48 Allow to create variables named _ 2024-11-20 14:55:05 +01:00
Guillaume Gomez
53f15d8931 Simplify Target::PlaceHolder handling 2024-11-19 21:31:05 +01:00
Guillaume Gomez
7aa8868b29 Extend checks for underscore ident misuses 2024-11-19 21:21:54 +01:00
Guillaume Gomez
f39a846598 Allow idents to start with _ 2024-11-19 18:10:06 +01:00
Guillaume Gomez
abe167effc Update book about filters 2024-11-18 15:15:21 +01:00
Guillaume Gomez
d7bc44a87d Remove restriction preventing to have | surrounded by whistespace characters 2024-11-17 20:25:01 +01:00
Ed Page
93f7788aa8 refactor(parser): Remove redundant closure 2024-11-14 13:36:43 -06:00
René Kijewski
ebdd3f8b43 parser: allow whitespace control on extends 2024-11-14 16:53:14 +01:00
René Kijewski
d3b9b6ea56 Raise MSRV to 1.80 2024-11-13 23:33:06 +01:00
René Kijewski
9b733d8887 parser: use built-in whitespace trimming
Our `is_ws()` implementation was the same since the earliest days of askama:
<4805acc8eb (diff-a9b78e3979c46af7f12ceef379a5a7a657fbe8d77900aa6b4e88b7519b415ca3R31-R34)>.
It considers spaces ` `, horizontal tabs `\t`, carriage return `\r`, and
end of line `\n` characters as white spaces characters. Rust's
`u8::is_ascii_whitespace()` adds formfeed `\f` to the list.

This PR replaces `is_ws()` with the built-in
`char::is_ascii_whitespace()`. The parser is a tiny bit faster, and we
are a bit more standard conforming:

<details>
<summary>benchmark results</summary>

```text
     Running benches/from_str.rs (/home/kijewski/git/rust/rinja/target/release/deps/from_str-fae58673a015eed7)
librustdoc/all          time:   [322.12 µs 323.71 µs 325.71 µs]
                        thrpt:  [43.355 MiB/s 43.623 MiB/s 43.837 MiB/s]
                 change:
                        time:   [-6.3192% -5.9056% -5.4469%] (p = 0.00 < 0.05)
                        thrpt:  [+5.7607% +6.2763% +6.7454%]
                        Performance has improved.
librustdoc/item_info    time:   [5.6035 µs 5.6117 µs 5.6220 µs]
                        thrpt:  [27.989 MiB/s 28.041 MiB/s 28.082 MiB/s]
                 change:
                        time:   [-5.7565% -5.2861% -4.7532%] (p = 0.00 < 0.05)
                        thrpt:  [+4.9904% +5.5811% +6.1081%]
                        Performance has improved.
librustdoc/item_union   time:   [32.750 µs 32.863 µs 32.984 µs]
                        thrpt:  [29.926 MiB/s 30.036 MiB/s 30.139 MiB/s]
                 change:
                        time:   [-8.0835% -7.2746% -6.3867%] (p = 0.00 < 0.05)
                        thrpt:  [+6.8224% +7.8453% +8.7944%]
                        Performance has improved.
librustdoc/page         time:   [149.16 µs 149.45 µs 149.84 µs]
                        thrpt:  [41.327 MiB/s 41.433 MiB/s 41.515 MiB/s]
                 change:
                        time:   [-7.1007% -6.8131% -6.5353%] (p = 0.00 < 0.05)
                        thrpt:  [+6.9922% +7.3113% +7.6435%]
                        Performance has improved.
librustdoc/print_item   time:   [18.142 µs 18.197 µs 18.253 µs]
                        thrpt:  [51.726 MiB/s 51.883 MiB/s 52.040 MiB/s]
                 change:
                        time:   [-5.4557% -5.0690% -4.6229%] (p = 0.00 < 0.05)
                        thrpt:  [+4.8470% +5.3396% +5.7706%]
                        Performance has improved.
librustdoc/short_item_info
                        time:   [17.533 µs 17.552 µs 17.576 µs]
                        thrpt:  [51.547 MiB/s 51.618 MiB/s 51.674 MiB/s]
                 change:
                        time:   [-6.0722% -5.6344% -5.2641%] (p = 0.00 < 0.05)
                        thrpt:  [+5.5566% +5.9708% +6.4647%]
                        Performance has improved.
librustdoc/sidebar      time:   [35.947 µs 36.056 µs 36.176 µs]
                        thrpt:  [34.113 MiB/s 34.226 MiB/s 34.330 MiB/s]
                 change:
                        time:   [-7.6516% -7.0295% -6.4676%] (p = 0.00 < 0.05)
                        thrpt:  [+6.9148% +7.5611% +8.2856%]
                        Performance has improved.
librustdoc/source       time:   [13.274 µs 13.318 µs 13.364 µs]
                        thrpt:  [55.161 MiB/s 55.353 MiB/s 55.537 MiB/s]
                 change:
                        time:   [-6.2263% -5.9307% -5.6552%] (p = 0.00 < 0.05)
                        thrpt:  [+5.9941% +6.3047% +6.6397%]
                        Performance has improved.
librustdoc/type_layout_size
                        time:   [8.1719 µs 8.2034 µs 8.2364 µs]
                        thrpt:  [32.884 MiB/s 33.016 MiB/s 33.143 MiB/s]
                 change:
                        time:   [-5.6332% -5.3531% -5.0853%] (p = 0.00 < 0.05)
                        thrpt:  [+5.3578% +5.6559% +5.9695%]
                        Performance has improved.
librustdoc/type_layout  time:   [29.034 µs 29.101 µs 29.175 µs]
                        thrpt:  [92.278 MiB/s 92.513 MiB/s 92.726 MiB/s]
                 change:
                        time:   [-8.3192% -7.9370% -7.4312%] (p = 0.00 < 0.05)
                        thrpt:  [+8.0278% +8.6212% +9.0741%]
                        Performance has improved.
```
</details>

I doubt that anyone is relying on the fact that the invisible character
`\f` won't be stripped by whitespace control. If you absolutely need
this character to be preserved, then you can always use `{{ '\u{b}' }}`.

Right now, `trim_ascii_start()` is manually implemented, because it is
only stable since rust 1.80.
2024-11-13 23:28:32 +01:00
René Kijewski
c5ed6af5a3 parser: add better error messages for broken closing block tags 2024-11-13 20:14:42 +01:00
René Kijewski
7f9c91de0b Add concat operator ~
The string concatenation operator `~` is present in jinja and tera:
<https://jinja.palletsprojects.com/en/stable/templates/#other-operators>.
While it's not the most important operator, it can come in handy e.g.
with filters: `{{ a|upper }}{{ b|upper }}` → `{{ (a ~ b)|upper }}`.
2024-11-12 17:07:17 +01:00
Ed Page
8cc7256c3f refactor(parser): Move off of deprecated re-export
The `sequence` mod is deprdcated.
2024-11-11 08:48:32 -06:00
René Kijewski
fdee5cfaf6 parser: simplify fuzzed_excessive_syntax_lengths 2024-11-10 22:18:02 +01:00
René Kijewski
9c53dcf7ff parser: reject excessively long syntax delimiters 2024-11-10 22:18:02 +01:00
René Kijewski
f8bb54bd96 Add failing test for excessively long syntax delimiters 2024-11-10 22:18:02 +01:00
Guillaume Gomez
e98cfa9ce8
Merge pull request #207 from GuillaumeGomez/macro-default-value
Add support for default value in macros
2024-11-09 19:18:49 +08:00
René Kijewski
19db2765b7 parser: more strict macro argument parsing 2024-11-09 08:37:54 +01:00
Ed Page
1ac0c35144
refactor(parser): Clean up prep for 0.5 (#226)
* refactor(parser): Rely on Parser trait rather than FnMut
* refactor(parser): Consistently use ParseResult
* refactor(parser): Accept Parser rather than FnMut
* refactor(parser): Chain the input through
* refactor(parser): Accept stateful Parser rather than stateless FnMut
* refactor(parser): Remove redundant closures
2024-11-07 20:54:48 +01:00
Ed Page
bafa30f2c4 refactor(parser): Switch 'impl FnMut' to 'impl Parser'
We did most of this in 88c702e but missed some spots
2024-10-28 10:10:36 -05:00
Ed Page
b08fbebf8f refactor(parser): Rely on Parser trait rather than FnMut
We did most of this in 25067bd but missed some spots.
2024-10-28 10:10:36 -05:00
Guillaume Gomez
b323289269 Turn check_duplicated_name into a function 2024-10-25 23:31:03 +02:00
Guillaume Gomez
6df0892d1a Make it possible to use macro arguments as default value 2024-10-25 23:24:11 +02:00
Guillaume Gomez
3c24304bba Add check for duplicated macro arguments 2024-10-25 23:21:05 +02:00
Guillaume Gomez
077f119916 Add support for default value in macros 2024-10-25 23:21:04 +02:00
Ed Page
4f5e1f2e6a refactor(parser): Move away from 'take_till*(str)' 2024-10-25 13:16:04 -05:00
Ed Page
817ae00364 perf(parser): Move away from 'one_of(str)'
Its slow and was removed in 0.5
2024-10-25 13:14:40 -05:00
Ed Page
b62b65072b refactor(parser): Express the intent with Char::parse 2024-10-25 13:13:15 -05:00
Ed Page
e56e3b663e refactor(parser): Simplify identifier parser 2024-10-25 13:12:15 -05:00
Ed Page
51322071fe refactor(parser): Simplify keyword parser 2024-10-25 13:11:43 -05:00