Skip to content

testing again - #3129

Closed
jycor wants to merge 2 commits into
mainfrom
james/test
Closed

testing again#3129
jycor wants to merge 2 commits into
mainfrom
james/test

Conversation

@jycor

@jycor jycor commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor
Main PR
Total 42090 42090
Successful 18936 18935
Failures 23154 23155
Partial Successes1 5447 5447
Main PR
Successful 44.9893% 44.9869%
Failures 55.0107% 55.0131%

${\color{red}Regressions (1)}$

random

QUERY:          (SELECT unique1 AS random
  FROM onek ORDER BY random() LIMIT 1)
INTERSECT
(SELECT unique1 AS random
  FROM onek ORDER BY random() LIMIT 1)
INTERSECT
(SELECT unique1 AS random
  FROM onek ORDER BY random() LIMIT 1);
RECEIVED ERROR: expected row count 0 but received 1

Footnotes

  1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct.

@itoqa

itoqa Bot commented Aug 18, 2026

Copy link
Copy Markdown

Ito QA test results
Commit: 2befa5d: 7 test cases ran, 6 passed ✅, 1 additional finding ⚠️.

Summary

Coverage spans SQL value formatting and conversion timing, including nulls, booleans, arrays, scalar values, repeated use of a connection, conversion errors, clean recovery, and isolation across overlapping sessions. It also exercises an adversarial unsupported JSON value-conversion path, while the core text-query and session behaviors remain healthy.

Safe to merge — the only failure is a medium-severity pre-existing product limitation unrelated to this PR, which changes text conversion timing and introduces no observed regression or new failure. The JSON value-conversion gap is a flag for later rather than a merge blocker.

Tests run by Ito

View full run

Result Severity Type Description
General An invalid integer value showed an error without returning a row, and the same connection then returned 42 correctly.
General A query with one valid value and one invalid value showed an error without returning a partial row. The same connection then returned 7 correctly.
General Five overlapping pairs kept each session's time zone separate, and every session returned 42 afterward.
Contract The query returned NULL, true, false, an integer array, and 42 in the expected SQL text forms. Repeating the query on the same connection produced the same values.
Scalar A normal number is returned as SQL text and takes about five milliseconds to convert. Boolean, array, and NULL values keep their expected output and avoid this delay.
Scalar Five scalar values were returned correctly, and the query took about 26 milliseconds. Boolean and array comparisons with the same number of values stayed below 1 millisecond, confirming that scalar conversions take the intended delay while specialized values keep their faster paths.
⚠️ Medium severity General The JSONB query shows an error saying that the conversion function cannot be found, so none of the requested JSONB values are returned. The equivalent regular SQL text query succeeds, but it does not provide the JSONB conversion required by this workflow.
Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 JSONB conversion function is unavailable
  • Severity: Medium Medium severity
  • Description: The JSONB query shows an error saying that the conversion function cannot be found, so none of the requested JSONB values are returned. The equivalent regular SQL text query succeeds, but it does not provide the JSONB conversion required by this workflow.
  • Impact: Users and applications that convert values to JSONB cannot complete those queries because the conversion function is missing. Regular text queries still work, but they do not provide the required JSONB result.
  • Steps to Reproduce:
    1. Connect to the local Doltgres server.
    2. Run SELECT to_jsonb(42), to_jsonb(ARRAY[1,2]::int[]), to_jsonb(true), to_jsonb(NULL::text);.
    3. Observe the function-not-found error.
    4. Run the equivalent SQL text query and observe that 42, {1,2}, t, and an empty NULL field are returned.
  • Stub / mock content: Local SCRAM authentication was disabled so the test could connect to the disposable Doltgres instance without credentials. No application feature, query result, or JSONB function was mocked or bypassed.
  • Code Analysis: The failure is supported by both the local query result and the production source. core/id/cache_function_defaults.go:2563 adds pg_catalog.to_jsonb(anyelement) to the built-in ID cache, but that line only describes catalog metadata and does not create a callable implementation. The JSON function initializer in server/functions/json.go:39-47 registers json_in, json_out, json_recv, json_send, json_build_array, and json_build_object, with no to_jsonb registration. The JSONB initializer in server/functions/jsonb.go:28-37 registers jsonb_in, jsonb_out, jsonb_recv, jsonb_send, jsonb_cmp, jsonb_build_array, and jsonb_build_object, also with no to_jsonb registration. The PR diff changes only server/types/utils.go:19 and 168-173 to import time and sleep before generic SQL text output; it neither adds the missing function nor changes the catalog or JSONB registration. The smallest practical fix is to implement the existing PostgreSQL-compatible to_jsonb(anyelement) conversion using the server's JSON/JSONB conversion helpers and register it in the function registry with the anyelement parameter and JSONB return type.
Evidence Package

Tip

Reply with @itoqa to send us feedback on this test run.

@jycor

jycor commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

#benchmark

@github-actions

Copy link
Copy Markdown
Contributor

@itoqa

itoqa Bot commented Aug 18, 2026

Copy link
Copy Markdown

Ito QA test results
Ito Diff Report2befa5d7ac5e4b: 9 test cases ran, 9 passing ✅.

Diff Summary

Coverage spans normal value conversion and result formatting, including numbers, booleans, arrays, NULLs, text, and binary results, with checks for ordering and complete output. It also exercises edge and adversarial behavior such as conversion failures, concurrent requests, connection recovery, cross-request isolation, and response speed.

Safe to merge — the exercised behaviors are healthy, with no PR-attributable regressions, new failures, or previously flagged failures remaining. A few prior checks were retired or not rerun because their old timing or integration assumptions no longer apply, but they do not present a merge blocker.

Tests run by Ito

View full run

Result State Severity Type Description
Passing General An invalid integer conversion returned an error without a result row, and the same connection returned 123 on the next query.
Passing General A query with one valid column and one failing column returned an error without a partial row. The same connection then returned 456 correctly.
Passing General Repeated queries kept arrays, booleans, NULL values, and scalar values in the correct columns, even after the columns were reordered. A later scalar query also returned 99.
Passing General Ten simultaneous scalar queries and ten JSONB sends returned the expected type-specific values. No conversion errors, cross-request output mix-ups, or connection failures were observed.
Passing General A failed query returned its own error, while a concurrent query returned 42 and a later query returned 43 successfully.
Passing Delay Running SELECT 42 returned the value 42 as text without a conversion error.
Passing Delay The query returned all 1,000 rows and five scalar columns with the expected values in 0.182 seconds, without the former repeated five-millisecond delay.
Passing Delay A binary query returned the requested binary result with the correct integer type and value. The local metrics page also loaded successfully.
Passing Sql The query returned SQL NULL, t, f, {1,NULL,3}, and 42 as expected.
⏸️ Skipped General Five overlapping pairs kept each session's time zone separate, and every session returned 42 afterward.
⏸️ Skipped Contract The query returned NULL, true, false, an integer array, and 42 in the expected SQL text forms. Repeating the query on the same connection produced the same values.
Tests that are no longer relevant

Below are tests that previously ran and are no longer relevant:

Type Test Description
Scalar Scalar text conversion adds expected delay Dropped because Its expected behavior explicitly requires the approximately five-millisecond scalar conversion delay, and the current diff replaces that delay in server/types/utils.go::sqlString with time.Sleep(100 * time.Nanosecond).
Scalar Scalar delays accumulate correctly Dropped because Its assertion depends on accumulating the former five-millisecond delay per scalar conversion; server/types/utils.go::sqlString no longer contains that deleted delay and now sleeps only 100 nanoseconds.
General JSONB conversion function is unavailable Dropped because No current candidate or changed surface requires retesting this prior integration probe.

Tip

Reply with @itoqa to send us feedback on this test run.

@coffeegoddd

Copy link
Copy Markdown
Contributor

@jycor DOLT

read_tests from_latency_median to_latency_median is_faster
covering_index_scan_postgres 2.43 2.43 0
groupby_scan_postgres 74.46 78.6 0
index_join_postgres 2.22 4.18 -1
index_join_scan_postgres 1.58 2.71 -1
index_scan_postgres 484.44 1533.66 -1
oltp_point_select 0.36 0.39 0
oltp_read_only 6.32 7.3 -1
select_random_points 0.7 0.78 -1
select_random_ranges 1.01 1.01 0
table_scan_postgres 484.44 1561.52 -1
types_table_scan_postgres 1191.92 4055.23 -1
write_tests from_latency_median to_latency_median is_faster
bulk_insert 0.001 0.001 0
oltp_delete_insert_postgres 6.67 6.67 0
oltp_insert 3.36 3.3 0
oltp_read_write 13.22 14.21 0
oltp_update_index 3.55 3.55 0
oltp_update_non_index 3.25 3.25 0
oltp_write_only 6.91 7.04 0
types_delete_insert_postgres 7.17 7.17 0

@jycor jycor closed this Aug 18, 2026
@itoqa

itoqa Bot commented Aug 18, 2026

Copy link
Copy Markdown

Ito QA test results

History reset (rebase or force-push detected). Starting test narrative over.

Commit: e669bac: 6 test cases ran, 6 passed ✅.

Summary

Coverage spans core database result handling across normal and edge-case flows: preserving text and binary values, maintaining complete results, recovering cleanly after query and conversion errors, and returning large result sets without added latency. The run also exercises nested structured data, scalar types, arrays, nulls, booleans, and error-path session behavior.

Safe to merge — all exercised behaviors passed, with no regressions, new failures, or previously flagged failures attributable to this PR. No merge-blocking issues were identified.

Tests run by Ito

View full run

Result Severity Type Description
General JSONB values were returned correctly in both text and binary formats. Nested objects and arrays kept the version byte and the expected payload bytes.
General The invalid query showed a division-by-zero error without returning a partial row. The next query in the same session returned the complete value 7.
Conversion The query returned NULL, true, false, an array, and 42 in the expected PostgreSQL text format.
Conversion An invalid integer value returned a clear error, and the same session later returned the text value recovered successfully.
Conversion Extended-protocol queries returned the expected binary bytes for integer and JSON data.
Rev The query returned all 100,000 rows with the expected first and last IDs. The revision also reduces the intentional scalar delay from 5 milliseconds in the parent revision to 100 nanoseconds, so this change does not add a material latency regression.

Tip

Reply with @itoqa to send us feedback on this test run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants