Skip to content

[WIP][SPARK-59144][SQL] Route plain SQL/JSON constructor calls through routine resolution - #58450

Open
ganeshashree wants to merge 1 commit into
apache:masterfrom
ganeshashree:SPARK-59144
Open

[WIP][SPARK-59144][SQL] Route plain SQL/JSON constructor calls through routine resolution#58450
ganeshashree wants to merge 1 commit into
apache:masterfrom
ganeshashree:SPARK-59144

Conversation

@ganeshashree

@ganeshashree ganeshashree commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

JSON_VALUE / JSON_QUERY / JSON_EXISTS / JSON_ARRAY each have a dedicated grammar branch before #functionCall, so a plain call with no constructor clause was built directly into the built-in expression, bypassing function resolution.

This PR routes the clause-free form through normal function resolution (emitting UnresolvedFunction) and registers the four names as built-ins that rebuild the expression with the standard clause defaults when unshadowed, mirroring EXTRACT.

  • Resolution precedence (SQL PATH shadowing, builtin.name / system.builtin.name, spark.sql.legacy.persistentCatalogFirst) is consolidated into FunctionResolution.functionNameResolvesToBuiltin and applied consistently in both the fixed-point analyzer and the single-pass resolver.
  • Star arguments (e.g. json_array(*)) are rejected for the built-ins, but expand normally when a routine shadows the name.
  • A JSON_ARRAY/JSON_QUERY that is a direct JSON_ARRAY element stays on the direct-construction path so its FORMAT JSON splicing is frozen lexically; a routed nested constructor carries its element format via a tree-node tag read through analyzer-inserted recoloring casts.

Why are the changes needed?

Because these constructors bypassed function resolution, a same-named routine on the SQL PATH could not shadow them, and the names were absent from FunctionRegistry. Routing them through routine resolution makes them behave like other built-ins for shadowing and registry lookup.

Does this PR introduce any user-facing change?

Yes.

  • A user-defined temp/persistent function named json_value, json_query, json_exists, or json_array can now shadow the built-in on the SQL PATH.
  • The four names now appear in FunctionRegistry (e.g. in SHOW FUNCTIONS).
  • Star arguments such as json_array(*) now raise the standard invalid-star-usage error for the built-ins instead of being handled by the dedicated grammar branch.

All changes are within the unreleased master/branch; there is no change relative to a released Spark version.

How was this patch tested?

Added and extended unit tests: JsonValueSuite, JsonQuerySuite, JsonExistsSuite, JsonArraySuite, SetPathSuite, ExpressionParserSuite, and ResolverGuardSuite, covering routing, SQL PATH shadowing, persistentCatalogFirst, star-argument rejection, and nested-constructor FORMAT JSON behavior. Updated the sql-expression-schema.md golden file for the newly registered functions.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

Follow-ups (out of scope for this PR)

This PR implements shadowing only — clause-free SQL/JSON constructor calls route
through routine resolution so a temp/persistent function shadows the built-in. Deferred:

  • Nested constructors through routed/qualified calls. A nested JSON constructor
    reaching a routed JSON_ARRAY (only the qualified spellings, e.g.
    builtin.json_array(json_array(1))) is currently quoted (["[1]"]), not spliced
    ([[1]]). Nesting via the JSON_ARRAY(...) grammar and unqualified
    json_array(json_array(1)) still splices. Splicing through a routed call — and the
    related object-level-collation recoloring consistency — is left as a follow-up, since
    the qualified spelling is new and not required for shadowing.
  • JsonValue.eval ExceptionNonFatal is an unrelated robustness fix; can be
    split into its own PR.
  • JsonPathExpressionBuilder foldable-path eval() is uncaught; wrapping it to emit
    a clean analysis error is low-priority hardening.

@ganeshashree ganeshashree changed the title [SPARK-59144][SQL] Route plain SQL/JSON constructor calls through routine resolution [WIP][SPARK-59144][SQL] Route plain SQL/JSON constructor calls through routine resolution Sep 2, 2026
@ganeshashree
ganeshashree force-pushed the SPARK-59144 branch 3 times, most recently from 6380840 to a047bc0 Compare September 2, 2026 07:42
…tine resolution

Route clause-free JSON_VALUE/JSON_QUERY/JSON_EXISTS/JSON_ARRAY calls through function
resolution (registered as built-ins) so a temporary or persistent function of the same
name can shadow the built-in; clause-bearing forms stay direct and are not shadowable.
Handle count(*) and reject star arguments in the routed forms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants