Skip to content

[global-index] Fix full-text range selection and scalar-filter coverage for multi-field indexes#8547

Open
CrownChu wants to merge 3 commits into
apache:masterfrom
CrownChu:feature-globalindex-common-core
Open

[global-index] Fix full-text range selection and scalar-filter coverage for multi-field indexes#8547
CrownChu wants to merge 3 commits into
apache:masterfrom
CrownChu:feature-globalindex-common-core

Conversation

@CrownChu

Copy link
Copy Markdown
Contributor

Purpose

A global index may contain one primary field and multiple extra fields. However, the current scan paths assume that:

  1. a full-text search column must be the primary index field; and
  2. a vector-primary index cannot also serve scalar predicates on its extra fields.
    These assumptions prevent multi-field indexes from serving full-text and scalar queries correctly. They may also cause indexed row ranges to be skipped when different index definitions cover different ranges of the same column.
    This PR makes the generic global-index scan and read paths multi-field aware:
  • Recognize full-text columns in both indexFieldId and extraFieldIds.
  • Select one index definition for each (column, row range).
  • Prefer a dedicated full-text index when multiple indexes cover the same range.
  • Preserve ranges that are covered only by an extra-field index.
  • Create the global indexer for each split from that split's own metadata.
  • Allow a vector-primary multi-field index to serve scalar predicates on its extra fields.
  • Preserve raw vector-search fallback when the vector column is not the primary field.
  • Prune redundant IS NOT NULL predicates only when another predicate on the same field is null-rejecting, preserving the semantics of expressions such as:
    field IS NULL AND field IS NOT NULL.
    The changes are backend-independent and do not introduce dependencies on any specific global-index implementation.

Tests

  • GlobalIndexEvaluatorTest: 19 tests passed.
  • Added and verified full-text search coverage for:
    • a text column stored as an extra field;
    • preferring a dedicated full-text index;
    • ranges covered only by an extra-field index;
    • indexes that do not contain the queried text column.
  • Relevant FullTextSearchBuilderTest cases: 4 tests passed.

@CrownChu CrownChu force-pushed the feature-globalindex-common-core branch from 970083e to b5ecfec Compare July 10, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant