* chore(sqlx-postgres): fix typo in `migrate.rs` comment
* feat(sqlite): support `no_tx` migrations
SQLite includes several SQL statements that are useful during migrations but
must be executed outside of a transaction to take effect, such as `PRAGMA
foreign_keys = ON|OFF` or `VACUUM`. Additionally, advanced migrations may want
more precise control over how statements are grouped into transactions or
savepoints to achieve the desired atomicity for different parts of the
migration.
While SQLx already supports marking migrations to run outside explicit
transactions through a `-- no-transaction` comment, this feature is currently
only available for `PgConnection`'s `Migrate` implementation, leaving SQLite and
MySQL without this capability. Although it's possible to work around this
limitation by implementing custom migration logic instead of executing
`Migrator#run`, this comes at a cost of significantly reduced developer
ergonomics: code that relies on the default migration logic, such as
`#[sqlx::test]` or `cargo sqlx database setup`, won't support these migrations.
These changes extend `SqliteConnection`'s `Migrate` implementation to support
`no_tx` migrations in the same way as PostgreSQL, addressing this feature gap. I
also considered implementing the same functionality for MySQL, but since I
haven't found a practical use case for it yet, and every
non-transaction-friendly statement I could think about in MySQL triggers
implicit commits anyway, I determined it wasn't necessary at this time and could
be considered an overreach.
* test(sqlite): add test for `no_tx` migrations
* chore(sqlx-sqlite): bring back useful comment
* chore(sqlx-sqlite): unify SQL dialect in annotation comments
* Allow single-field named structs to be transparent
This more closely matches the criteria for e.g. #[repr(transparent)]
and #[serde(transparent)].
* Add tests, fix error messages
In the previous code the worker would always assume that the custom command worked. However the higher level code would run a check and notice that a transaction was not actually started and raise an error without rolling back the transaction.
This improves the code by moving the transaction check into the worker to ensure that the transaction depth tracker is only modified if the user's custom command actually started a transaction.
Fixes: https://github.com/launchbadge/sqlx/issues/3932
* feat: Implement `get_transaction_depth` for drivers
* test: Verify `get_transaction_depth()` on postgres
* Refactor: `TransactionManager` delegation without BC
SQLite implementation is currently WIP
* Fix: Avoid breaking changes on `AnyConnectionBackend`
* Refactor: Remove verbose `SqliteConnection` typing
* Feat: Implementation for SQLite
I have included `AtomicUsize` in `WorkerSharedState`. Ideally, it is not desirable to execute `load` and `fetch_add` in two separate steps, but we decided to allow it here since there is only one thread writing. To prevent writing from other threads, the field itself was made private, and a getter method was provided with `pub(crate)`.
* Refactor: Same approach for `cached_statements_size`
ref: a66787d36d62876b55475ef2326d17bade817aed
* Fix: Add missing `is_in_transaction` for backend
* Doc: Remove verbose "synchronously" word
* Fix: Remove useless `mut` qualifier
* feat: add Connection::begin_with
This patch completes the plumbing of an optional statement from these methods to
`TransactionManager::begin` without any validation of the provided statement.
There is a new `Error::InvalidSavePoint` which is triggered by any attempt to
call `Connection::begin_with` when we are already inside of a transaction.
* feat: add Pool::begin_with and Pool::try_begin_with
* feat: add Error::BeginFailed and validate that custom "begin" statements are successful
* chore: add tests of Error::BeginFailed
* chore: add tests of Error::InvalidSavePointStatement
* chore: test begin_with works for all SQLite "BEGIN" statements
* chore: improve comment on Connection::begin_with
* feat: add default impl of `Connection::begin_with`
This makes the new method a non-breaking change.
* refactor: combine if statement + unwrap_or_else into one match
* feat: use in-memory SQLite DB to avoid conflicts across tests run in parallel
* feedback: remove public wrapper for sqlite3_txn_state
Move the wrapper directly into the test that uses it instead.
* fix: cache Status on MySqlConnection
* fix: compilation errors
* fix: format
* fix: postgres test
* refactor: delete `Connection::get_transaction_depth`
* fix: tests
---------
Co-authored-by: mpyw <ryosuke_i_628@yahoo.co.jp>
Co-authored-by: Duncan Fairbanks <duncanfairbanks6@gmail.com>
* feat: implement serialze no copy on lockedsqlitehandle
* feat: implement serialize on sqliteconnection
* feat: implement deserialize on sqliteconnection and add sqlitebuf wrapper type
* refactor: misc sqlite type and deserialize refactoring
* chore: misc clippy refactoring
* fix: misc refactoring and fixes
- pass non-owned byte slice to deserialize
- `SqliteBufError` and better error handling
- more impl for `SqliteOnwedBuf` so it can be used as a slice
- default serialize for `SqliteConnection`
* refactor: move serialize and deserialize on worker thread
This implements `Command::Serialize` and `Command::Deserialize` and moves the
serialize and deserialize logic to the worker thread.
`Serialize` will need some more iterations as it's not clear whether it would
need to wait for other write transactions before running.
* refactor: misc refactoring and changes
- Merged deserialize module with serialize module
- Moved `SqliteOwnedBuf` into serialize module
- Fixed rustdocs
* chore: API tweaks, better docs, tests
* fix: unused import
* fix: export `SqliteOwnedBuf`, docs and safety tweaks
---------
Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
* fix: fixture macro attribute
* remove extra new line
* add extra new line
* feat: add test for slqx::test macro
* feat: update test for sqlx::test macro
* remove old macro test
* feat: add postgres and sqlite test
* rust format
* cargo fmt
* fix fixtures execution order in test
* feat: Add set_update_hook on SqliteConnection
* refactor: Address PR comments
* fix: Expose UpdateHookResult for public use
---------
Co-authored-by: John Smith <asserta4@gmail.com>
* convert logger to output a query graph
* avoid duplicating branch paths to shrink output graph
* separate different branching paths
* include all branches which found unique states
* track the reason for ending each branches execution
* track the result type of each branch
* make edges rely on history index instead of program_id, to avoid errors when looping
* add state diff to query graph
* drop redundant table info
* rework graph to show state changes, rework logger to store state snapshots
* show state on the previous operation
* gather duplicate state changes into clusters to reduce repetition
* draw invisible connections between unknown instructions by program_i
* clean up dot format string escaping
* add test case from #1960 (update returning all columns)
* add tests for #2939 (update returning only the PK column)
* allow inserting into a table using only the index
* improve null handling of IfNull, fix output type of NewRowId
* add NoResult nodes for branches which don't log a result, as a sanity check
* add short-circuit to all logging operations
* remove duplicate logging checks, and make logging enabled/disabled consistently depend on sqlx::explain instead of sqlx for capture & sqlx::explain for output
* add failing test for awkwardly nested/filtered count subquery
* handle special case of return operation to fix failing test
* require trace log level instead of using whatever log level statement logging was configured to use
Breaking changes:
* integer decoding will now loudly error on overflow instead of silently truncating.
* some usages of the query!() macros might change an i32 to an i64.
Also adds support for *decoding* `u64`s because there's no reason not to.
Closes#3179
This is meant to be much easier to discover than the current approach of directly invoking `Executor` methods.
In addition, I'm improving documentation for the `query*()` functions across the board.
* chore:Added ipaddr extension library to gitignore
* fix:In a Linux environment, shared libraries in the current directory are not loaded, so add the current directory to the LD_LIBRARY_PATH environment variable.
* fix: Since confrict primary key when running multiple sqlite tests, removed specific primary key in insert.
* chore: Since avoid git modified targeting, copy the db file to new test db file.
* fix: Since docker mysql 5.7 using yaSSL(It only supports TLSv1.1), avoid running when using rustls.
Inlined format args make code more readable, and code more compact.
I ran this clippy command to fix most cases, and then cleaned up a few trailing commas and uncaught edge cases.
```
cargo clippy --bins --examples --benches --tests --lib --workspace --fix -- -A clippy::all -W clippy::uninlined_format_args
```
* Add failing tests
* remove unnecessary functions, clarify function names
* simplify access to cursor columns with helper methods
* split table info from cursor info so that cursors can share table info
* fix test expectations
* use native-tls API
* Add client cert and key to MySQL connector
* Add client ssl tests for PostgreSQL
* Add client ssl tests for MariaDB and MySQL
* Adapt GA tests
* Fix RUSTFLAGS to run all tests
* Remove containers to free the DB port before running SSL auth tests
* Fix CI bad naming
* Use docker-compose down to remove also the network
* Fix main rebase
* Stop trying to stop service using docker-compose, simply use docker cmd
* Fix RUSTFLAGS for Postgres
* Name the Docker images for MariaDB and MySQL so we can stop them using their name
* Add the exception for mysql 5.7 not supporting compatible TLS version with RusTLS
* Rebase fixes
* Set correctly tls struct (fix merge)
* Handle Elliptic Curve variant for private key
* Fix tests suite
* Fix features in CI
* Add tests for Postgres 15 + rebase
* Python tests: fix exception for MySQL 5.7 + remove unneeded for loops
* CI: run SSL tests only when building with TLS support
---------
Co-authored-by: Barry Simons <linuxuser586@gmail.com>
* add failing test for nested orderby
* log query paths which were abandoned due to invalid state or looping. Allow instructions to be executed a small number of times to fix nested order by query
* add failing testcase using nested orderby
* fix handling of sequence/offset and rewind
* fix handling when sqlite nests records inside of records
* add test of temporary table handling
* WIP add test failure for temp table access
* fix support for temp tables
* add tests for sqlite datetime functions
* add basic date and time function support
* handle gosub opcode correctly
* add group by test
* fix group by handling
* add additional passing group by test
* add test case for simple limit query
* fix IfPos & If touching wrong branches state, fix IfPos using wrong branch criteria
* add test for large offsets
* add short-circuit for possible query offset loops
* add groupby query that is predicted incorrectly
* fix handling of integer cast failures
* add tests for single-row aggregate results
* fix handling of null-based branching
* add test for coercion of text by sum
* fix calculation of sum value coercion
* add failing test for recursive with query
* add logic for delete operation to fix queries grouping by columns from a recursive query
* add failing test cases for update/delete return into
* fix regression in null tracking by improving tracking of cursor empty/full state
* add failing test case for order by column types
* Add support for SorterOpen,SorterInsert,SorterData
* add failing test case for unions
* fix range copy/move implementation
* fix wrong copy/move range
* remove calls to dbg!
* fix: ensure migration progress is not lost for PG
Fixes#1966.
* fix: ensure migration progress is not lost for sqlite
This is similar to #1966.
* fix: ensure reverse migration progress is not lost for PG
See #1966.
* fix: ensure reverse migration progress is not lost for sqlite
See #1966.
* fix: ensure migration progress is not lost for mysql
This is similar to #1966.
* fix: ensure reverse migration progress is not lost for mysql
See #1966.
* test: check migration type as well
* test: extend migrations testing
* fix: work around MySQL implicit commits
* refactor: simplify migration testing
* Add extension support for SQLite
While SQLite supports loading extensions at run-time via either the C
API or the SQL interface, they strongly recommend [1] only enabling the C
API so that SQL injections don't allow attackers to run arbitrary
extension code.
Here we take the most conservative approach, we enable only the C
function, and then only when the user requests extensions be loaded in
their `SqliteConnectOptions`, and disable it again once we're done
loading those requested modules. We don't add any support for loading
extensions via environment variables or connection strings.
Extensions in the options are stored as an IndexMap as the load order
can have side effects, they will be loaded in the order they are
supplied by the caller.
Extensions with custom entry points are supported, but a default API
is exposed as most users will interact with extensions using the
defaults.
[1]: https://sqlite.org/c3ref/enable_load_extension.html
* Add extension testing for SQlite
Extends x.py to download an appropriate shared object file for supported
operating systems, and uses wget to fetch one into the GitHub Actions
context for use by CI.
Overriding LD_LIBRARY_PATH for only this specific DB minimises the
impact on the rest of the suite.
* SQLite: Execute SQLCipher pragmas as very first operations on the database
SQLCipher requires, apart from 'key' pragma also other cipher-related
to be executed before read/write to the database.
* Added tests for SQLCipher functionality
* remove default-features from libsqlite3-sys when building from dev-dependencies
Co-authored-by: Szymon Zimnowoda <szimnowoda.memri@gmail.com>
* use direct blocking calls for SQLite in `sqlx_macros`
* this also ensures the database is closed properly, cleaning up tempfiles
* don't send `PRAGMA journal_mode` unless set
* this previously defaulted to WAL mode which is a permanent setting
on databases which doesn't necessarily apply to all use-cases
* changing into or out of WAL mode acquires an exclusive lock on the database
that can't be waited on by `sqlite3_busy_timeout()`
* for consistency, `sqlx-cli` commands that create databases will still
create SQLite databases in WAL mode; added a flag to disable this.
* in general, don't send `PRAGMA`s unless different than default
* we were sending a bunch of `PRAGMA`s with their default values just to enforce
an execution order on them, but we can also do this by inserting empty slots
for their keys into the `IndexMap`
* add error code to `SqliteError` printout
* document why `u64` is not supported
* feat(sqlite): Add 'time' crate support for date/time types
docs(sqlite): Update types module docs for JSON and Chrono
docs(mysql): Update types module docs for JSON
* More efficient time crate decoding with FormatItem::First and hand-crafting of format descriptions
* Replace temporary testing code with original intention
* Replace duplicated formatting test with intended test
* Performance improvements to decoding OffsetDateTime, PrimitiveDateTime, and Time
* Use correct iteration for OffsetDateTime
* Reduce visibility of format constants
Co-authored-by: John B Codes <johnbcodes@users.noreply.github.com>
