56 Commits

Author SHA1 Message Date
René Kijewski
3775f4e3a3 Use unicode-ident to determine what is an identifier
Resolves <https://github.com/askama-rs/askama/issues/442>.

The performance is actually slightly better than before. `unicode-ident`
is highly optimized and jump free.

One test from a fuzzer outcome had to be deleted, because it contained
identifiers that weren't actually identifiers. There is still a test
that tests the same problem, but every identifier is simply `x`. In
another fuzzed test a character [`U+E0049`] was removed.

[`U+E0049`]: https://en.wikipedia.org/w/index.php?oldid=1278382889

<details>
<summary>Benchmark results</summary>

```text
$ cd askama_parser && cargo bench

librustdoc/all          time:   [184.47 µs 185.30 µs 186.08 µs]
                        thrpt:  [75.887 MiB/s 76.207 MiB/s 76.548 MiB/s]
                 change:
                        time:   [−1.4364% −0.9672% −0.4738%] (p = 0.00 < 0.05)
                        thrpt:  [+0.4761% +0.9767% +1.4573%]
                        Change within noise threshold.

librustdoc/item_info    time:   [3.3880 µs 3.3892 µs 3.3906 µs]
                        thrpt:  [46.409 MiB/s 46.428 MiB/s 46.445 MiB/s]
                 change:
                        time:   [−3.8735% −3.5927% −3.2949%] (p = 0.00 < 0.05)
                        thrpt:  [+3.4071% +3.7266% +4.0296%]
                        Performance has improved.

librustdoc/item_union   time:   [20.052 µs 20.087 µs 20.126 µs]
                        thrpt:  [49.044 MiB/s 49.140 MiB/s 49.224 MiB/s]
                 change:
                        time:   [−2.2419% −1.8647% −1.5113%] (p = 0.00 < 0.05)
                        thrpt:  [+1.5345% +1.9002% +2.2933%]
                        Performance has improved.

librustdoc/page         time:   [85.828 µs 86.157 µs 86.518 µs]
                        thrpt:  [71.571 MiB/s 71.871 MiB/s 72.147 MiB/s]
                 change:
                        time:   [−1.2728% −0.7668% −0.2512%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2518% +0.7727% +1.2892%]
                        Change within noise threshold.

librustdoc/print_item   time:   [10.065 µs 10.101 µs 10.138 µs]
                        thrpt:  [93.132 MiB/s 93.469 MiB/s 93.806 MiB/s]
                 change:
                        time:   [−3.3793% −2.8352% −2.3267%] (p = 0.00 < 0.05)
                        thrpt:  [+2.3821% +2.9180% +3.4975%]
                        Performance has improved.

librustdoc/short_item_info
                        time:   [9.0741 µs 9.1018 µs 9.1377 µs]
                        thrpt:  [99.148 MiB/s 99.540 MiB/s 99.843 MiB/s]
                 change:
                        time:   [−4.7480% −4.2335% −3.7763%] (p = 0.00 < 0.05)
                        thrpt:  [+3.9245% +4.4207% +4.9847%]
                        Performance has improved.

librustdoc/sidebar      time:   [21.468 µs 21.555 µs 21.648 µs]
                        thrpt:  [57.004 MiB/s 57.252 MiB/s 57.482 MiB/s]
                 change:
                        time:   [−3.7641% −3.0465% −2.4191%] (p = 0.00 < 0.05)
                        thrpt:  [+2.4791% +3.1423% +3.9114%]
                        Performance has improved.

librustdoc/source       time:   [7.9602 µs 7.9780 µs 7.9929 µs]
                        thrpt:  [92.230 MiB/s 92.403 MiB/s 92.609 MiB/s]
                 change:
                        time:   [−1.6386% −1.0684% −0.5875%] (p = 0.00 < 0.05)
                        thrpt:  [+0.5910% +1.0799% +1.6659%]
                        Change within noise threshold.

librustdoc/type_layout_size
                        time:   [4.7821 µs 4.7915 µs 4.8017 µs]
                        thrpt:  [56.406 MiB/s 56.526 MiB/s 56.637 MiB/s]
                 change:
                        time:   [−1.9743% −1.4867% −1.0153%] (p = 0.00 < 0.05)
                        thrpt:  [+1.0257% +1.5091% +2.0141%]
                        Performance has improved.

librustdoc/type_layout  time:   [15.022 µs 15.051 µs 15.076 µs]
                        thrpt:  [178.57 MiB/s 178.88 MiB/s 179.22 MiB/s]
                 change:
                        time:   [−1.5028% −1.0358% −0.5705%] (p = 0.00 < 0.05)
                        thrpt:  [+0.5738% +1.0466% +1.5257%]
                        Change within noise threshold.
```
</details>
2025-05-18 23:04:47 +02:00
René Kijewski
ab798e2482 Allow filters with explicit path
IMHO that makes custom filters much easier to use.
2025-05-17 13:20:54 +02:00
Guillaume Gomez
6a16256f24 Fix new clippy lints 2025-04-22 11:34:55 +02:00
René Kijewski
b402936db3 generator: add named arguments for filters 2025-04-21 23:11:24 +02:00
Guillaume Gomez
3b52c907c3 Rename rinja into askama 2025-03-11 19:31:06 +01:00
Guillaume Gomez
2d80cc3524 Rename everything to rinja 2024-06-17 15:58:52 +02:00
Dirkjan Ochtman
1170d54c3b parser: improve parse error structure 2024-06-07 12:12:15 +02:00
Dirkjan Ochtman
627d58bc3b parser: simplify ErrorContext construction 2024-05-21 11:26:21 +02:00
Dirkjan Ochtman
c3582f5d89 parser: attach unclosed error creation to ErrorContext 2024-05-21 11:26:21 +02:00
Dirkjan Ochtman
cbe8ac3c9e parser: order inherent impl before trait impls 2024-05-21 11:26:21 +02:00
Guillaume Gomez
c6d0ba8dbc Add checks for chars 2024-04-26 16:50:12 +02:00
Guillaume Gomez
3e5ccaeea5 Add more tests for floating parsing 2024-04-24 14:46:43 +02:00
vallentin
c1d945a882 Moved cfg not windows 2024-04-21 14:35:21 +02:00
Guillaume Gomez
0b4b7d6345 Improve error message by providing the file where the error occurred if it is not an inlined source 2024-04-05 12:13:54 +02:00
Guillaume Gomez
5338f62a93 Add support for filter chaining in filter blocks 2024-03-11 14:25:46 +01:00
Guillaume Gomez
9439638e0f Create new Filter type 2024-03-11 14:25:46 +01:00
René Kijewski
7aa4a80791 Remove redundant import 2024-03-07 17:10:58 +01:00
Guillaume Gomez
279105bd4d Remove unused Next variant value 2024-03-07 14:55:53 +01:00
Dirkjan Ochtman
79738ff238 Fix support for mixed case variables 2024-01-18 11:23:18 +01:00
max
2c7759c0d0 Improve performance of find_used_templates
Signed-off-by: max <gmx.sht@gmail.com>
2024-01-17 17:58:53 +01:00
max
7f30a657f6 Undo an accidental deletion of #[derive(Debug)]
Signed-off-by: max <gmx.sht@gmail.com>
2023-12-07 10:17:43 +01:00
max
33eb70c34a Added testing for reserved variable names
Signed-off-by: max <gmx.sht@gmail.com>
2023-12-07 10:17:43 +01:00
PizzasBear
696561003d
Add better support for rust-like number literals (#908)
Signed-off-by: max <gmx.sht@gmail.com>
2023-11-22 16:09:33 +01:00
Guillaume Gomez
76462255ea Create a type alias for all IResult to simplify code reading 2023-10-25 16:26:36 +02:00
Guillaume Gomez
c3281e353d Improve error for invalid name used in endblock 2023-10-25 14:30:35 +02:00
Guillaume Gomez
6dae920a45 Create ErrorContext type 2023-10-25 14:30:35 +02:00
René Kijewski
eef38cea67 Pass Node parsing level to expressions 2023-09-28 17:28:26 +02:00
René Kijewski
238e4bbad7 Limit nesting in parser nodes, too 2023-09-28 17:28:26 +02:00
Dirkjan Ochtman
5718175291 Move Level into the crate root 2023-09-28 17:28:26 +02:00
René Kijewski
95ff27c087 Parse paths and identifiers only once
In the old implementation each variable in an expression would be parsed
up to three times:

* Try to parse a path because it contains a leading double colon, or
  infix double colons.
* Try to parse it as path again by scanning for an identifier that
  contains an upper case character.
* Fall back to scanning for any identifier.

This PR turns all three steps into one, without the need for
backtracking.
2023-08-03 00:02:04 +02:00
René Kijewski
06ad86b1e1 parser: allow negative numbers in patterns 2023-08-02 11:09:05 +02:00
René Kijewski
d09041bd0a parser: better error message for unknown nodes 2023-08-02 11:09:05 +02:00
René Kijewski
cdfe287b73 parser: remove re-exports parser::{node,expr}::* 2023-08-01 13:04:41 +02:00
René Kijewski
1521a08a20 parser: add type for Node::Extends 2023-08-01 13:04:41 +02:00
René Kijewski
14df8e16c1 parser: add type for Node::Include 2023-08-01 13:04:41 +02:00
René Kijewski
c5c46ada40 parser: rename Node::Cond into If and add type 2023-08-01 13:04:41 +02:00
René Kijewski
fa3a3271ee parser: add type for Node::Let 2023-08-01 13:04:41 +02:00
René Kijewski
6323b5e171 parser: add type for Node::Raw 2023-08-01 13:04:41 +02:00
René Kijewski
7b7c38b377 parser: add type for Node::Lit 2023-08-01 13:04:41 +02:00
René Kijewski
26f598c0d1 parser: add type for Node::BlockDef 2023-08-01 13:04:41 +02:00
René Kijewski
108c4a6a33 parser: add type for Node::Match 2023-08-01 13:04:41 +02:00
René Kijewski
4f52dbe8be parser: add type for Node::Call 2023-08-01 13:04:41 +02:00
René Kijewski
cd075630ea parser: add type for Node::Import 2023-08-01 13:04:41 +02:00
René Kijewski
ffe267be7a parser: add Ast::nodes() method 2023-08-01 10:51:57 +02:00
René Kijewski
cbf766fa1f parser: impement PartialEq for Ast 2023-08-01 10:51:57 +02:00
René Kijewski
cfed9274e2 parser: implement Debug for Parsed 2023-08-01 10:51:57 +02:00
René Kijewski
382c3239f2 parser: use ? 2023-08-01 10:51:57 +02:00
René Kijewski
b3843d90a8 parser: ensure correct drop order for Parsed
According to [RFC 1857] the fields of a struct are dropped in the same
order as they are declared. For `struct S { a: A, b: B }` field `a` is
dropped before field `b`.

Our struct `Parsed` is self referencial. Its field `ast` depends on
`source`, so `source` must outlife `ast`. This PR changes the order of
the fields to reflect this requirement. In practice it should not
matter, because we know that the variant of `Node` won't access the
string data during their `Drop`, but better safe than sorry - maybe
`Node` changes in the future.

[RFC 1857]: https://rust-lang.github.io/rfcs/1857-stabilize-drop-order.html
2023-08-01 10:51:57 +02:00
Dirkjan Ochtman
cdbe8ef39f parser: move node parsers into impl block 2023-07-31 10:27:15 +02:00
Dirkjan Ochtman
dcfa5e1c69 parser: simplify top-level parser match 2023-07-31 10:27:15 +02:00