[GLUTEN-12722][CH] Support stack function

[exmy]

What changes are proposed in this pull request?

Support stack function

close #12722

How was this patch tested?

Add ut.

Was this patch authored or co-authored using generative AI tooling?

Codex

Related issue: #12722

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[Copilot]

Pull request overview

Adds ClickHouse-backend support for Spark’s stack generator so it can be planned and executed natively (offloaded), and introduces unit coverage to ensure both correctness and offload behavior across supported Spark versions.

Changes:

• Implement stack handling in ClickHouse local-engine ExpressionParser by rewriting it into array construction + arrayJoin (and tuple extraction when needed).
• Unblock stack expression validation for ClickHouse by removing it from the scalar-function blacklist.
• Add Gluten UTs (Spark 3.3–4.1 variants) to validate stack results and assert the query plan is offloaded via GenerateExecTransformerBase.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
gluten-ut/spark41/src/test/scala/org/apache/spark/sql/GlutenGeneratorFunctionSuite.scala	Adds stack correctness + offload assertion test for Spark 4.1 UTs.
gluten-ut/spark40/src/test/scala/org/apache/spark/sql/GlutenGeneratorFunctionSuite.scala	Adds stack correctness + offload assertion test for Spark 4.0 UTs.
gluten-ut/spark35/src/test/scala/org/apache/spark/sql/GlutenGeneratorFunctionSuite.scala	Adds stack correctness + offload assertion test for Spark 3.5 UTs.
gluten-ut/spark34/src/test/scala/org/apache/spark/sql/GlutenGeneratorFunctionSuite.scala	Introduces stack offload test in previously-empty suite for Spark 3.4 UTs.
gluten-ut/spark33/src/test/scala/org/apache/spark/sql/GlutenGeneratorFunctionSuite.scala	Introduces stack offload test in previously-empty suite for Spark 3.3 UTs.
cpp-ch/local-engine/Parser/ExpressionParser.h	Declares parseStack entry point in the ClickHouse expression parser.
cpp-ch/local-engine/Parser/ExpressionParser.cpp	Implements parseStack and wires it into expressionsToActionsDAG dispatch for signature "stack".
backends-clickhouse/src/main/scala/org/apache/gluten/utils/CHExpressionUtil.scala	Removes STACK from the ClickHouse scalar-function blacklist to allow offload.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

[lgbo-ustc]

Could you add a test case without null padding, e.g. stack(2, id, id + 1, id + 2, id + 3)? The current test only covers the padding path where the last field is filled with NULL. A no-padding case can verify that the row/field mapping is correct when all stack cells are populated.

[lgbo-ustc]

Could you add more detailed comments for ExpressionParser::parseStack to explain the implementation idea? The logic is a bit non-trivial: it reorganizes Spark stack arguments into per-output-field arrays, pads missing cells with NULL via arrayResize, combines fields with arrayZip, then uses arrayJoin and sparkTupleElement to produce the generator outputs. A short explanation in the code would make this easier to maintain and review later.

[github-actions]

Run Gluten Clickhouse CI on x86

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (1)

cpp-ch/local-engine/Parser/ExpressionParser.cpp:900

• parseStack has a dedicated num_fields == 1 execution path (skips arrayZip and returns the arrayJoin result directly), but the added UTs only cover the multi-field case (num_fields == 2). Please add a unit test that exercises a single-output-column stack (e.g., stack(2, id, id + 1) and/or a padding case like stack(3, id, id + 1)) to ensure this branch produces correct rows and remains offloaded.

    const DB::ActionsDAG::Node * array_join_input;
    if (num_fields == 1)
        array_join_input = field_arrays[0];
    else
        array_join_input = toFunctionNode(actions_dag, "arrayZip", field_arrays);

    array_join_input = &actions_dag.materializeNode(*array_join_input);
    const auto * array_join_node = &actions_dag.addArrayJoin(*array_join_input, array_join_input->result_name);

    if (num_fields == 1)
    {
        actions_dag.addOrReplaceInOutputs(*array_join_node);
        return {array_join_node};
    }

[github-actions]

Run Gluten Clickhouse CI on x86

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.