[WIP][SPARK-59144][SQL] Route plain SQL/JSON constructor calls through routine resolution - #58450
Open
ganeshashree wants to merge 1 commit into
Open
[WIP][SPARK-59144][SQL] Route plain SQL/JSON constructor calls through routine resolution#58450ganeshashree wants to merge 1 commit into
ganeshashree wants to merge 1 commit into
Conversation
ganeshashree
force-pushed
the
SPARK-59144
branch
from
September 1, 2026 12:57
f0d2cdb to
0864659
Compare
uros-b
reviewed
Sep 1, 2026
ganeshashree
force-pushed
the
SPARK-59144
branch
3 times, most recently
from
September 2, 2026 07:42
6380840 to
a047bc0
Compare
…tine resolution Route clause-free JSON_VALUE/JSON_QUERY/JSON_EXISTS/JSON_ARRAY calls through function resolution (registered as built-ins) so a temporary or persistent function of the same name can shadow the built-in; clause-bearing forms stay direct and are not shadowable. Handle count(*) and reject star arguments in the routed forms.
ganeshashree
force-pushed
the
SPARK-59144
branch
from
September 2, 2026 16:33
a047bc0 to
1fed0fa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
JSON_VALUE/JSON_QUERY/JSON_EXISTS/JSON_ARRAYeach have a dedicated grammar branch before#functionCall, so a plain call with no constructor clause was built directly into the built-in expression, bypassing function resolution.This PR routes the clause-free form through normal function resolution (emitting
UnresolvedFunction) and registers the four names as built-ins that rebuild the expression with the standard clause defaults when unshadowed, mirroringEXTRACT.builtin.name/system.builtin.name,spark.sql.legacy.persistentCatalogFirst) is consolidated intoFunctionResolution.functionNameResolvesToBuiltinand applied consistently in both the fixed-point analyzer and the single-pass resolver.json_array(*)) are rejected for the built-ins, but expand normally when a routine shadows the name.JSON_ARRAY/JSON_QUERYthat is a directJSON_ARRAYelement stays on the direct-construction path so itsFORMAT JSONsplicing is frozen lexically; a routed nested constructor carries its element format via a tree-node tag read through analyzer-inserted recoloring casts.Why are the changes needed?
Because these constructors bypassed function resolution, a same-named routine on the SQL PATH could not shadow them, and the names were absent from
FunctionRegistry. Routing them through routine resolution makes them behave like other built-ins for shadowing and registry lookup.Does this PR introduce any user-facing change?
Yes.
json_value,json_query,json_exists, orjson_arraycan now shadow the built-in on the SQL PATH.FunctionRegistry(e.g. inSHOW FUNCTIONS).json_array(*)now raise the standard invalid-star-usage error for the built-ins instead of being handled by the dedicated grammar branch.All changes are within the unreleased master/branch; there is no change relative to a released Spark version.
How was this patch tested?
Added and extended unit tests:
JsonValueSuite,JsonQuerySuite,JsonExistsSuite,JsonArraySuite,SetPathSuite,ExpressionParserSuite, andResolverGuardSuite, covering routing, SQL PATH shadowing,persistentCatalogFirst, star-argument rejection, and nested-constructorFORMAT JSONbehavior. Updated thesql-expression-schema.mdgolden file for the newly registered functions.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Follow-ups (out of scope for this PR)
This PR implements shadowing only — clause-free SQL/JSON constructor calls route
through routine resolution so a temp/persistent function shadows the built-in. Deferred:
reaching a routed
JSON_ARRAY(only the qualified spellings, e.g.builtin.json_array(json_array(1))) is currently quoted (["[1]"]), not spliced(
[[1]]). Nesting via theJSON_ARRAY(...)grammar and unqualifiedjson_array(json_array(1))still splices. Splicing through a routed call — and therelated object-level-collation recoloring consistency — is left as a follow-up, since
the qualified spelling is new and not required for shadowing.
JsonValue.evalException→NonFatalis an unrelated robustness fix; can besplit into its own PR.
JsonPathExpressionBuilderfoldable-patheval()is uncaught; wrapping it to emita clean analysis error is low-priority hardening.