You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b66aab5
Browse filesBrowse the repository at this point in the historyBrowse files
Fix bugs found by the independent pre-release check; E2E on every supported Ruby (#49)
## Summary
Fixes for the issues found by an independent pre-release check of `main`
(7194e3b) against real PostgreSQL 16, MySQL 8.4 and SQLite on Ruby
3.1–4.0. Each fix comes with E2E coverage that fails on `main`.
**Fixes**
- **`Timeout` inside `transaction` committed the partial work on Ruby
3.1/3.2 (all drivers).** timeout < 0.4 exits the block with `throw`,
which `transaction` treated as success.
- For `rubyVersion` < 3.3, the generated `transaction` now records
normal completion. If the block exits without completing or raising
while the loaded timeout is < 0.4 (`timeout_exits_with_throw?`), it
rolls back.
- The generated Gemfile adds `gem 'timeout', '>= 0.4'`.
- Output for `rubyVersion` >= 3.3 is unchanged.
- **SQLite numbered parameters were bound in the wrong order.** sqlc
writes `sqlc.arg` as `?N` on SQLite, but the generated code bound params
positionally.
- `sqlc.slice` followed by a named arg raised `RangeException`.
- Mixing repeated `sqlc.arg` with `?` silently returned wrong rows. This
also affected `:batch*`.
- The sqlite3 driver now scans the SQL, skipping literals and comments,
and rewrites `?N` to `?`. It binds params in occurrence order, repeating
values when a param appears more than once.
- **pg: json/jsonb parameters `true`/`false` failed.** `encode_json` now
JSON-encodes booleans as well as Arrays.
- **Query names `Raise`/`Rand`/`Format` shadowed Kernel methods** the
generated class calls. They are now generation errors
(`Code::KERNEL_METHODS`). A unit test scans the generated examples and
golden files for unreserved bare Kernel calls.
- **Keyword order.** Required keywords now come before optional ones
(`def m(lim:, name_filter: nil)`), so generated code stays RuboCop-clean
(`Style/KeywordParametersOrder`). Binding order is unchanged.
- **README.**
- The fiber-scheduler claim now says mysql2 prepared statements block
the scheduler; use threads.
- The Transactions section documents the timeout behaviour and when
parent-object calls reuse the transaction's connection.
**Test coverage**
- E2E now runs on Ruby 3.1, 3.2, 3.3, 3.4 and 4.0 for all three engines.
- On 3.1/3.2, `scripts/tests/generate_examples.rb` regenerates the
examples for that `rubyVersion` (Struct rows) into a scratch dir
(`EXAMPLES_DIR`), and the jobs install from `Gemfile.e2e`.
- The tests build rows with keywords, so they work for both `Data` and
`Struct`.
- New E2E cases:
- transactions: Timeout during a DB call and during Ruby code;
`break`/`return`/`throw` semantics; `Thread#kill`; nested savepoint
rollback; parent-object calls inside a transaction
- pg: deferred FK failing at COMMIT, terminated backend
- mysql2: KILLed connection
- JSON value types
- SQLite mixed placeholders (direct and `:batchone`)
- named `:copyfrom`
- generated-method concurrency (pg under `Sync`, mysql2 across threads)
- seeded randomized `:copyfrom`, batch and concurrent-transaction plans
- New examples queries cover the cases above. Fixtures were recaptured
(`scripts/tests/capture_fixtures.rb`) and goldens updated.
**Follow-up in this PR: mysql2 Fiber concurrency and full E2E coverage**
- **mysql2 queries now run concurrently under `Sync`.**
- The mysql2 driver now quotes parameters client-side (`quote`, using
`Mysql2::Client#escape`; hex literals for binary strings; unsupported
types raise `TypeError`). This is the same approach as ActiveRecord's
mysql2 adapter.
- It binds them with a literal- and comment-aware `bind_params` and
sends one `Client#query`. There are no server-side prepared statements
anymore.
- E2E tests cover the concurrency (generated methods under `Sync`),
quoting and injection, and the absence of prepared statements.
- **E2E coverage of every documented feature.** `docs/Coverage.md` maps
each README claim to the E2E test(s) per engine. A unit test fails if a
referenced test does not exist.
- **New CI jobs.**
- Option-variant E2E jobs: `moduleName: MyApp::Db` for all engines, plus
pg `preparedStatements: false` running the whole pg suite.
- `tests/end2end_options.rb` (rename, exactTableNames, overrides, global
options, several query files, `generateTypes: false`,
generated-Gemfile-only load).
- The 3.1/3.2 jobs install from the generated Gemfiles.
- **New checks inside the suites.**
- Automatic runtime RBS checks of every value returned by generated
methods.
- Row frozen / pattern-matching assertions.
- More types and engine-specific gaps: isolation actually applied,
SQLite lock modes, logger, enums, names, outer-join embeds, batch
non-atomicity.
- **Testing policy** in CLAUDE.md and docs/Testing.md: every supported
feature needs an E2E test per engine (listed in docs/Coverage.md); bug
fixes start with a failing E2E test; unit tests only where E2E cannot
reach.
## Checklist
- [x] `examples/` is regenerated (`rake sqlc_generate_process`) and
committed
- [x] Tests added or updated (`rake test`, and `rake end2end_*` when the
generated code changes)
- [x] The generated code stays RuboCop-clean (`rake rubocop`) and
Steep-clean (`rake steep`)
- [x] Labelled (`bug`, `enhancement`, `documentation`, or `breaking`) so
the release notes are grouped correctly
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Doron Eli Rachman <doron050@gmail.com>
***Generated code must stay valid for Ruby 3.1–4.0** (`Struct` below 3.2, `Data` from 3.2) and RuboCop/Steep-clean.
32
32
* Generated code is plain, idiomatic Ruby (keyword args, `Data` rows, implicit returns, 2-space indent); do not mirror
33
33
the C# sibling plugin's style. Parameter *values* are never logged.
34
-
* Prepared statements are per connection (`ConnectionState`) because the query object is shared by pooled connections.
34
+
* PostgreSQL prepared statements are per connection (`ConnectionState`) because the query object is shared by pooled
35
+
connections. MySQL uses client-side quoting and `Mysql2::Client#query` instead of server-side prepared statements.
36
+
* Every README claim or supported feature needs a real-database E2E test for each applicable engine, listed in
37
+
`docs/Coverage.md`. A bug fix starts with an E2E test that fails before the fix; use unit tests only where E2E cannot
38
+
reach the behavior (for example option validation or generation errors).
39
+
* E2E coverage runs on every supported Ruby and includes full-suite option variants. Update `docs/Coverage.md` and
40
+
`docs/Testing.md` with new supported behavior and its test names.
35
41
* sqlc runs process plugins with an empty env (`env: [PATH]` is needed in `sqlc.local.yaml`) and resolves `out:`
36
42
relative to the config file.
37
43
38
44
## Git, CI and releases
39
45
* Develop on a feature branch, small commits; PRs target `main`. Label PRs (`bug`, `enhancement`, `documentation`; `breaking` for incompatible changes, see `.github/release.yml`).
40
-
* CI: `build.yml` (WASM, lint, `sqlc diff`, Steep, option matrix) and `tests.yml` (unit, Ruby 3.1/3.2/3.4/4.0, E2E on real
`mode:`(`sqlite3`) takes `:deferred`, `:immediate`, `:exclusive`. Other values raise before anything runs, and so
121
128
do `isolation:` / `mode:` on a nested `transaction` (a savepoint cannot change them).
@@ -170,7 +177,11 @@ server; duplicate keys and invalid values raise `Mysql2::Error` instead of being
170
177
The generated SQLite code uses `Monitor`: if you run Steep, add `library "monitor"` to your `Steepfile`.
171
178
172
179
### Names
173
-
* Names are snake_cased, acronyms kept together (`MIN_ID` → `min_id`, `HTTPServer` → `http_server`). Ruby keywords and names a row or method already uses get a trailing `_` (`class_`, `hash_`).
180
+
* Query methods and keyword arguments are snake_cased, with acronyms kept together (`MIN_ID` → `min_id`,
181
+
`HTTPServer`→ `http_server`); Ruby keywords get a trailing `_` (`end_:`). Row members use the column spelling
182
+
reported by sqlc for the engine, except names that conflict with generated row methods (`class_`, `hash_`):
183
+
PostgreSQL and SQLite keep `"camelCase"` as `:camelCase`, while MySQL reports it as `:camelcase`. A column named
184
+
`end`is the row member `:end` but the argument `end_:`.
174
185
* A parameter used twice is one keyword when named with `sqlc.arg`; repeated `?` / `$n` of one column become `id:`, `id_2:`.
175
186
* A table `authors` next to a table `author` keeps its plural model name (`Authors`). Query names that would replace
176
187
a generated method (`Transaction`, `Initialize`, ...) are errors.
@@ -179,8 +190,8 @@ server; duplicate keys and invalid values raise `Mysql2::Error` instead of being
179
190
* Two result columns with the same name (e.g. `a.id, b.id`) are an error: alias one (`b.id AS book_id`) or use `sqlc.embed`.
180
191
* SQLite's `NUMERIC` affinity can return an `Integer` where `Float` is declared.
181
192
* `:batchexec` / `:batchone` / `:batchmany` are not atomic on their own: wrap them in `transaction` for all-or-nothing.
182
-
* SQLite: a transaction belongs to the thread that opened it; a Fiber / `Enumerator#next` inside it that uses the same
183
-
database waits for it (with `async`, run the work in the transaction's own fiber).
193
+
* SQLite transactions belong to the thread that opened them; fibers sharing the same database take turns, and a
194
+
transaction holds it until it ends.
184
195
* Not applicable to Ruby (they exist in the C# plugin): `withAsyncSuffix` / `withCancellationToken` (use the Fiber
|`type_resolver.rb`| RBS type of a column / parameter: driver type table, arrays, nullability (`T?`), `overrides:` (a `db_type` override types array elements too) |
28
28
|`catalog.rb`| the schema's tables and enums, and their Ruby names |
29
29
|`drivers/base.rb`| the driver interface and what all drivers share: `instrument` logging, `NotFound`, transaction RBS |
30
-
|`drivers/{pg,mysql2,sqlite3}.rb`| the per-library parts: type mapping, connecting / pooling, preparing and executing statements, transactions, `:copyfrom`|
30
+
|`drivers/{pg,mysql2,sqlite3}.rb`| the per-library parts: type mapping, connecting / pooling, executing statements, transactions, `:copyfrom`|
31
31
|`code.rb`| text helpers: indentation, `snake_case`, `camelize`, Ruby-keyword-safe parameter names, `rubocop:disable` pairs for schema names that break a naming cop |
32
32
33
33
**Rule of thumb:**`MethodGen` / `Generator` decide *what* is generated and are the same for every database; a driver only
@@ -62,11 +62,13 @@ Look at [`examples/pg/query_sql.rb`](../examples/pg/query_sql.rb) for a complete
62
62
63
63
Design decisions worth knowing (the *why* is in the comments of `drivers/base.rb` and `drivers/pg.rb`):
64
64
65
-
* **Prepared statements are per connection.** A pooled query object is shared by many connections, and a prepared
66
-
statement exists only on the connection that prepared it, so each connection remembers its own (`ConnectionState`).
65
+
* **PostgreSQL prepared statements are per connection.** A pooled query object is shared by many connections, and a
66
+
prepared statement exists only on the connection that prepared it, so each connection remembers its own
67
+
(`ConnectionState`). MySQL quotes values client-side and uses `Mysql2::Client#query` so queries cooperate with the
68
+
Fiber scheduler.
67
69
***Pool injection.**`QuerySql.new(pool)` accepts anything that responds to `#with`, so an app can share its pool
68
70
and tests can pass a fake. `connect(...)` is only a convenience that builds a `connection_pool`.
69
-
***Statementsare closed.**SQLite/MySQL statements are closed in`ensure`(an openSQLitestatement blocks `COMMIT`).
71
+
***SQLite statements are closed.**Theyare closed in`ensure`because an open statement blocks `COMMIT`.
70
72
***Parameters are encoded by the driver.**`Driver#encode_param` (e.g. `bytea`, `json`and arrays on `pg`) is used by
71
73
every command, batches included; add a type's encoding there, not in `MethodGen`.
72
74
* **`:copyfrom` is all or nothing** on every driver (a savepoint / transaction around the load), and MySQL's
0 commit comments