44 Commits

Author SHA1 Message Date
iamjpotts
ff93aa017a
refactor(sqlite): do not borrow bound values, delete lifetime on SqliteArguments (#3957)
* bug(sqlite): query macro argument lifetime use inconsistent with other db platforms

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>

* refactor(sqlite): Improve support for references as query macro bind arguments by removing lifetime parameter from SqliteArguments

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>

* refactor(sqlite): Introduce SqliteArgumentsBuffer type

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>

* refactor(sqlite): Improve cloning of SqliteArgumentValue, SqliteArguments, and SqliteArgumentsBuffer

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>

* refactor(any): Simplify AnyArguments::convert_to to convert_into

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>

---------

Signed-off-by: Joshua Potts <8704475+iamjpotts@users.noreply.github.com>
2025-08-19 13:59:37 -07:00
Kevin Cox
1f7af3abc2
SQLite: fix transaction level accounting with bad custom command. (#3981)
In the previous code the worker would always assume that the custom command worked. However the higher level code would run a check and notice that a transaction was not actually started and raise an error without rolling back the transaction.

This improves the code by moving the transaction check into the worker to ensure that the transaction depth tracker is only modified if the user's custom command actually started a transaction.

Fixes: https://github.com/launchbadge/sqlx/issues/3932
2025-08-18 16:16:52 -07:00
Austin Bonander
21598cfec6
breaking(sqlite): libsqlite3-sys versioning, feature flags, safety changes (#3928) 2025-07-17 01:13:32 -07:00
Joey de Waal
469f22788e
breaking: add SqlStr (#3723)
* refactor: introduce `SqlSafeStr` API

* rebase main

* Add SqlStr + remove Statement lifetime

* Update the definition of Executor and AnyConnectionBackend + update Postgres driver

* Update MySql driver

* Update Sqlite driver

* remove debug clone count

* Reduce the amount of SqlStr clones

* improve QueryBuilder error message

* cargo fmt

* fix clippy warnings

* fix doc test

* Avoid panic in `QueryBuilder::reset`

* Use `QueryBuilder` when removing all test db's

* Add comment to `SqlStr`

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>

* Update sqlx-core/src/query_builder.rs

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>

* Add `Clone` as supertrait to `Statement`

* Move `Connection`, `AnyConnectionBackend` and `TransactionManager` to `SqlStr`

* Replace `sql_cloned` with `sql` in `Statement`

* Update `Executor` trait

* Update unit tests + QueryBuilder changes

* Remove code in comments

* Update comment in `QueryBuilder`

* Fix clippy warnings

* Update `Migrate` comment

* Small changes

* Move `Migration` to `SqlStr`

---------

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
2025-07-07 00:35:54 -07:00
Joey de Waal
0f891a3c56
Remove unnecessary boxfutures (#3525)
* Remove `BoxFuture`'s from `Connection` and `ConnectOptions`

* Remove `BoxFuture`'s for `MigrateDatabase`

* Remove `BoxFuture`'s for `TransactionManager`

* Remove `BoxFuture`'s for `TestSupport`

* Remove `BoxFuture`'s from `PgPoolCopyExt`

* Clippy fixes

* Box timeout future in debug mode
2025-07-04 17:53:46 -07:00
Austin Bonander
25cbeedab4
feat: create sqlx.toml format (#3383)
* feat: create `sqlx.toml` format

* feat: add support for ignored_chars config to sqlx_core::migrate

* chore: test ignored_chars with `U+FEFF` (ZWNBSP/BOM)

https://en.wikipedia.org/wiki/Byte_order_mark

* refactor: make `Config` always compiled

simplifies usage while still making parsing optional for less generated code

* refactor: add origin information to `Column`

* feat(macros): implement `type_override` and `column_override` from `sqlx.toml`

* refactor(sqlx.toml): make all keys kebab-case, create `macros.preferred-crates`

* feat: make macros aware of `macros.preferred-crates`

* feat: make `sqlx-cli` aware of `database-url-var`

* feat: teach macros about `migrate.table-name`, `migrations-dir`

* feat: teach macros about `migrate.ignored-chars`

* chore: delete unused source file `sqlx-cli/src/migration.rs`

* feat: teach `sqlx-cli` about `migrate.defaults`

* feat: teach `sqlx-cli` about `migrate.migrations-dir`

* feat: teach `sqlx-cli` about `migrate.table-name`

* feat: introduce `migrate.create-schemas`

* WIP feat: create multi-tenant database example

* fix(postgres): don't fetch `ColumnOrigin` for transparently-prepared statements

* feat: progress on axum-multi-tenant example

* feat(config): better errors for mislabeled fields

* WIP feat: filling out axum-multi-tenant example

* feat: multi-tenant example

No longer Axum-based because filling out the request routes would have distracted from the purpose of the example.

* chore(ci): test multi-tenant example

* fixup after merge

* fix(ci): enable `sqlx-toml` in CLI build for examples

* fix: CI, README for `multi-tenant`

* fix: clippy warnings

* fix: multi-tenant README

* fix: sequential versioning inference for migrations

* fix: migration versioning with explicit overrides

* fix: only warn on ambiguous crates if the invocation relies on it

* fix: remove unused imports

* fix: doctest

* fix: `sqlx mig add` behavior and tests

* fix: restore original type-checking order

* fix: deprecation warning in `tests/postgres/macros.rs`

* feat: create postgres/multi-database example

* fix: examples/postgres/multi-database

* fix: cargo fmt

* chore: add tests for config `migrate.defaults`

* fix: sqlx-cli/tests/add.rs

* feat(cli): add `--config` override to all relevant commands

* chore: run `sqlx mig add` test with `RUST_BACKTRACE=1`

* fix: properly canonicalize config path for `sqlx mig add` test

* fix: get `sqlx mig add` test passing

* fix(cli): test `migrate.ignored-chars`, fix bugs

* feat: create `macros.preferred-crates` example

* fix(examples): use workspace `sqlx`

* fix: examples

* fix(sqlite): unexpected feature flags in `type_checking.rs`

* fix: run `cargo fmt`

* fix: more example fixes

* fix(ci): preferred-crates setup

* fix(examples): enable default-features for workspace `sqlx`

* fix(examples): issues in `preferred-crates`

* chore: adjust error message for missing param type in `query!()`

* doc: mention new `sqlx.toml` configuration

* chore: add `CHANGELOG` entry

Normally I generate these when cutting the release, but I wanted to take time to editorialize this one.

* doc: fix new example titles

* refactor: make `sqlx-toml` feature non-default, improve errors

* refactor: eliminate panics in `Config` read path

* chore: remove unused `axum` dependency from new examples

* fix(config): restore fallback to default config for macros

* chore(config): remove use of `once_cell` (to match `main`)
2025-06-30 16:34:46 -07:00
Paolo Barbolini
df47ffedd2
Replace some more futures_util APIs with std variants (#3874) 2025-06-15 15:18:39 -07:00
Austin Bonander
90797200ee
groundwork for 0.9.0-alpha.1 (#3821)
* chore: bump version to `0.9.0-alpha.1`

* chore: delete unused `sqlx-bench` package

* chore: set `rust-version` to 1.85 for all crates

* fix: lots of new Clippy warnings

* fix: lots more Clippy warnings

* fix(cli): add `_sqlite` feature

* fix: lots, *lots* more Clippy warnings

* fix(core): warning in `tls_rustls`

* breaking: delete runtime+TLS combination features

* chore: don't re-export unstable `TransactionManager` trait

* chore: 0.9.0-alplha.1 CHANGELOG

* chore: increase MSRV further to 1.86

* fix: more clippy warnings
2025-06-01 21:09:55 -07:00
Cristian Le
97bf270caf
Always set SQLITE_OPEN_URI (#3289) 2025-04-13 17:06:17 -07:00
Austin Bonander
393b731d5e
Merge of #3427 (by @mpyw) and #3614 (by @bonsairobo) (#3765)
* feat: Implement `get_transaction_depth` for drivers

* test: Verify `get_transaction_depth()` on postgres

* Refactor: `TransactionManager` delegation without BC

SQLite implementation is currently WIP

* Fix: Avoid breaking changes on `AnyConnectionBackend`

* Refactor: Remove verbose `SqliteConnection` typing

* Feat: Implementation for SQLite

I have included `AtomicUsize` in `WorkerSharedState`. Ideally, it is not desirable to execute `load` and `fetch_add` in two separate steps, but we decided to allow it here since there is only one thread writing. To prevent writing from other threads, the field itself was made private, and a getter method was provided with `pub(crate)`.

* Refactor: Same approach for `cached_statements_size`

ref: a66787d36d62876b55475ef2326d17bade817aed

* Fix: Add missing `is_in_transaction` for backend

* Doc: Remove verbose "synchronously" word

* Fix: Remove useless `mut` qualifier

* feat: add Connection::begin_with

This patch completes the plumbing of an optional statement from these methods to
`TransactionManager::begin` without any validation of the provided statement.

There is a new `Error::InvalidSavePoint` which is triggered by any attempt to
call `Connection::begin_with` when we are already inside of a transaction.

* feat: add Pool::begin_with and Pool::try_begin_with

* feat: add Error::BeginFailed and validate that custom "begin" statements are successful

* chore: add tests of Error::BeginFailed

* chore: add tests of Error::InvalidSavePointStatement

* chore: test begin_with works for all SQLite "BEGIN" statements

* chore: improve comment on Connection::begin_with

* feat: add default impl of `Connection::begin_with`

This makes the new method a non-breaking change.

* refactor: combine if statement + unwrap_or_else into one match

* feat: use in-memory SQLite DB to avoid conflicts across tests run in parallel

* feedback: remove public wrapper for sqlite3_txn_state

Move the wrapper directly into the test that uses it instead.

* fix: cache Status on MySqlConnection

* fix: compilation errors

* fix: format

* fix: postgres test

* refactor: delete `Connection::get_transaction_depth`

* fix: tests

---------

Co-authored-by: mpyw <ryosuke_i_628@yahoo.co.jp>
Co-authored-by: Duncan Fairbanks <duncanfairbanks6@gmail.com>
2025-03-10 14:29:46 -07:00
Mattia Righetti
c5ea6c4435
feat: sqlx sqlite expose de/serialize (#3745)
* feat: implement serialze no copy on lockedsqlitehandle

* feat: implement serialize on sqliteconnection

* feat: implement deserialize on sqliteconnection and add sqlitebuf wrapper type

* refactor: misc sqlite type and deserialize refactoring

* chore: misc clippy refactoring

* fix: misc refactoring and fixes

- pass non-owned byte slice to deserialize
- `SqliteBufError` and better error handling
- more impl for `SqliteOnwedBuf` so it can be used as a slice
- default serialize for `SqliteConnection`

* refactor: move serialize and deserialize on worker thread

This implements `Command::Serialize` and `Command::Deserialize` and moves the
serialize and deserialize logic to the worker thread.

`Serialize` will need some more iterations as it's not clear whether it would
need to wait for other write transactions before running.

* refactor: misc refactoring and changes

- Merged deserialize module with serialize module
- Moved `SqliteOwnedBuf` into serialize module
- Fixed rustdocs

* chore: API tweaks, better docs, tests

* fix: unused import

* fix: export `SqliteOwnedBuf`, docs and safety tweaks

---------

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
2025-03-02 14:29:29 -08:00
Paolo Barbolini
65229f7ff9
Replace some futures_util APIs with std variants (#3721) 2025-02-01 16:01:56 -08:00
joeydewaal
546ec960a9
feat(Sqlite): add LockedSqliteHandle::last_error (#3707) 2025-01-27 20:56:21 -08:00
joeydewaal
f4c7498523
fix(Sqlite): stop sending rows after first error (#3700) 2025-01-24 14:41:29 -08:00
Austin Schey
aae800090b
feat(sqlite): add preupdate hook (#3625)
* feat: add preupdate hook

* address some PR comments

* add SqliteValueRef variant that takes a borrowed sqlite value pointer

* add PhantomData for additional lifetime check
2025-01-23 16:19:45 -08:00
gridbox
daeb87bef1
Add sqlite commit and rollback hooks (#3500)
* fix: Derive clone for SqliteOperation

* feat: Add sqlite commit and rollback hooks

---------

Co-authored-by: John Smith <asserta4@gmail.com>
2024-09-12 11:57:02 -07:00
Austin Bonander
394a7e86a7 fix(sqlite): fix unit and doctests 2024-08-23 23:39:32 -07:00
Austin Bonander
781b659352 fix(sqlite): audit for bad casts 2024-08-23 23:39:32 -07:00
Clark Kampfe
ff0252d4c2
sqlite: fix inconsistent read-after-write (#3354)
* sqlite: fix inconsistent read-after-write

fetch_one/fetch_optional

* try pushing fetch_optional early-return into worker

* run cargo fmt

* fix "it_can_execute_multiple_statements" test failure

* use Option<usize> instead of bespoke enum for rows returned
2024-08-01 13:27:01 -07:00
Jason Ish
c50572eef2
fix: sqlite update_hook char types (#3288)
Use "c_char" instead of "i8" as it resolves to the correct string type
on different architectures.

Fixes: #3287
2024-07-22 15:26:47 -07:00
Austin Bonander
16e3f1025a fix(postgres): add missing type resolution for arrays by name 2024-07-08 00:14:19 -07:00
Austin Bonander
bae083cf79 fix: clippy warnings 2024-06-13 13:11:36 -07:00
gridbox
0ea90881c1
feat: Add set_update_hook on SqliteConnection (#3260)
* feat: Add set_update_hook on SqliteConnection

* refactor: Address PR comments

* fix: Expose UpdateHookResult for public use

---------

Co-authored-by: John Smith <asserta4@gmail.com>
2024-06-05 19:06:15 -07:00
tyrelr
f960d5bc3b
Sqlite explain graph (#3064)
* convert logger to output a query graph

* avoid duplicating branch paths to shrink output graph

* separate different branching paths

* include all branches which found unique states

* track the reason for ending each branches execution

* track the result type of each branch

* make edges rely on history index instead of program_id, to avoid errors when looping

* add state diff to query graph

* drop redundant table info

* rework graph to show state changes, rework logger to store state snapshots

* show state on the previous operation

* gather duplicate state changes into clusters to reduce repetition

* draw invisible connections between unknown instructions by program_i

* clean up dot format string escaping

* add test case from #1960 (update returning all columns)

* add tests for #2939 (update returning only the PK column)

* allow inserting into a table using only the index

* improve null handling of IfNull, fix output type of NewRowId

* add NoResult nodes for branches which don't log a result, as a sanity check

* add short-circuit to all logging operations

* remove duplicate logging checks, and make logging enabled/disabled consistently depend on sqlx::explain instead of sqlx for capture & sqlx::explain for output

* add failing test for awkwardly nested/filtered count subquery

* handle special case of return operation to fix failing test

* require trace log level instead of using whatever log level  statement logging was configured to use
2024-05-31 12:57:54 -07:00
Max Bruckner
c57b46ceb6
Make Encode return a result (#3126)
* Make encode and encode_by_ref fallible

This only changes the trait for now and makes it compile, calling .expect() on all users. Those will be removed in a later commit.

* PgNumeric: Turn TryFrom Decimal to an infallible From

* Turn panics in Encode implementations into errors

* Add Encode error analogous to the Decode error

* Propagate decode errors through Arguments::add

This pushes the panics one level further to mostly bind calls. Those will also be removed later.

* Only check argument encoding at the end

* Use Result in Query internally

* Implement query_with functions in terms of _with_result

* Surface encode errors when executing a query.

* Remove remaining panics in AnyConnectionBackend implementations

* PostgreSQL BigDecimal: Return encode error immediately

* Arguments: Add len method to report how many arguments were added

* Query::bind: Report which argument failed to encode

* IsNull: Add is_null method

* MySqlArguments: Replace manual bitmap code with NullBitMap helper type

* Roll back buffer in MySqlArguments if encoding fails

* Roll back buffer in SqliteArguments if encoding fails

* Roll back PgArgumentBuffer if encoding fails
2024-05-31 12:42:36 -07:00
Austin Bonander
25efb2f7f4
breaking(sqlite): always use i64 as intermediate when decoding (#3184)
Breaking changes:

* integer decoding will now loudly error on overflow instead of silently truncating.
* some usages of the query!() macros might change an i32 to an i64.

Also adds support for *decoding* `u64`s because there's no reason not to.

Closes #3179
2024-04-13 16:59:13 -07:00
Paolo Barbolini
e0a72cf8f2
Drop urlencoding dependency (#3162) 2024-03-30 16:17:04 -07:00
Austin Bonander
59d5ba606e
fix(sqlite): delete unused ConnectionHandleRaw type (#3146) 2024-03-22 22:15:37 -07:00
zoomiti
a1e4984c6c
fix: spans in sqlite tracing (#2876) (#3056) 2024-02-15 22:27:22 -08:00
Austin Bonander
ea8e0ab9c4
fix(sqlite): use AtomicUsize for thread IDs (#2864) 2023-11-07 11:58:15 -08:00
Uttarayan Mondal
b85b72355e
[fix] Urlencode when passing filenames to sqlite3 (#2655) 2023-10-17 11:44:25 -07:00
Yuri Astrakhan
a824e8468c
Cleanup format arguments (#2650)
Inlined format args make code more readable, and code more compact.

I ran this clippy command to fix most cases, and then cleaned up a few trailing commas and uncaught edge cases.

```
cargo clippy --bins --examples  --benches --tests --lib --workspace --fix -- -A clippy::all -W clippy::uninlined_format_args
```
2023-07-31 13:27:04 -07:00
tyrelr
238a95b0f3
Sqlite analytical (#2508)
* Add failing tests

* remove unnecessary functions, clarify function names

* simplify access to cursor columns with helper methods

* split table info from cursor info so that cursors can share table info

* fix test expectations
2023-06-12 12:48:32 -07:00
tyrelr
0dfebb202f
Add and improve sqlite describe performance benchmarks (#2467)
* add basic describe benchmarks

* separate memory from query state

* move branch tracking & deduplication logic into a dedicated BranchList class

* Convert to using IntMap

* move intmap to a separate module, drop dead code, clean up function names

* Update Cargo.lock

* skip branches to check foreign keys, as they generally shouldn't impact query result type
2023-05-08 12:44:28 -07:00
tyrelr
39acaf1459
limit the number of instructions that can be evaluated (#2459)
* limit the number of instructions that can be evaluated

* code cleanup
2023-05-04 11:46:11 -07:00
Nisheeth Barthwal
4f1ac1d606
add progress handler support to sqlite (#2256)
* rebase main

* fmt

* use NonNull to fix UB

* apply code suggestions

* add test for multiple handler drops

* remove nightly features for test
2023-03-24 14:27:16 -07:00
Alexander Lyon
bd3eb94737 fix(#2407): respect the HaltIfNull opcode when determining nullability 2023-03-20 14:12:20 -07:00
Austin Bonander
d43257e18a chore: cleanup, fix all non-deprecation warnings 2023-03-16 17:03:14 -07:00
Austin Bonander
c4b835c23a feat: add Connection::shrink_buffers, PoolConnection::close
closes #2372
2023-03-03 16:50:27 -08:00
miles
1fd05716af [SQLite] Add option to execute PRAGMA optimize; on close of a connection (#2116)
* CHANGELOG: mention that users should upgrade CLI

* [SQLite] Add option to execute `PRAGMA optimize;` on close of a connection

* Update sqlx-sqlite/src/options/mod.rs

* Update sqlx-sqlite/src/options/mod.rs

* Update sqlx-sqlite/src/options/mod.rs

---------

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
2023-02-21 13:25:25 -08:00
Trangar
ad8ef8d608 Added regexp support in sqlite (#2189)
* CHANGELOG: mention that users should upgrade CLI

* Added regexp support in sqlite

* Added a with_regexp function to sqliteconnectoptions

* Fixed tests

* Undo CHANGELOG.md change

---------

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
Co-authored-by: Victor Koenders <victor.koenders@qrtech.se>
2023-02-21 13:25:25 -08:00
CosmicHorror
acaee75a30 Initial work to switch to tracing (#2185)
* Add tracing dep

* Switch over basic events

* Switch over dynamically enabled events

* Fix missing SocketAddr formatting

* More format fixing

* refactor: Apply tracing changes to new crate structure
2023-02-21 13:25:25 -08:00
tyrelr
5378dea6af Sqlite describe fixes (#2253)
* add failing test for nested orderby

* log query paths which were abandoned due to invalid state or looping.  Allow instructions to be executed a small number of times to fix nested order by query

* add failing testcase using nested orderby

* fix handling of sequence/offset and rewind

* fix handling when sqlite nests records inside of records

* add test of temporary table handling

* WIP add test failure for temp table access

* fix support for temp tables

* add tests for sqlite datetime functions

* add basic date and time function support

* handle gosub opcode correctly

* add group by test

* fix group by handling

* add additional passing group by test

* add test case for simple limit query

* fix IfPos & If touching wrong branches state, fix IfPos using wrong branch criteria

* add test for large offsets

* add short-circuit for possible query offset loops

* add groupby query that is predicted incorrectly

* fix handling of integer cast failures

* add tests for single-row aggregate results

* fix handling of null-based branching

* add test for coercion of text by sum

* fix calculation of sum value coercion

* add failing test for recursive with query

* add logic for delete operation to fix queries grouping by columns from a recursive query
2023-02-21 13:25:25 -08:00
Austin Bonander
b5312c3b6f Break drivers out into separate crates, clean up some technical debt (#2039)
* WIP rt refactors

* refactor: break drivers out into separate crates

also cleans up significant technical debt
2023-02-21 13:25:25 -08:00