Skip to content

test(api): replace MyScale and Elasticsearch JA test mocks with recording fakes - #44913

Closed
ShousenZHANG wants to merge 8 commits into
langgenius:mainfrom
ShousenZHANG:refactor/myscale-es-ja-test-fakes
Closed

ShousenZHANG wants to merge 8 commits into
langgenius:mainfrom
ShousenZHANG:refactor/myscale-es-ja-test-fakes

Conversation

@ShousenZHANG

@ShousenZHANG ShousenZHANG commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Part of #36288. Following #44775, #44853 and #44898, type the MyScale and Elasticsearch JA unit tests and replace their MagicMock objects with real values and small recording fakes.

  • test_myscale_vector.py imports the real provider module instead of reloading it under a stub clickhouse_connect module; clickhouse-connect is a dependency of dify-vdb-myscale. Only get_client is replaced: FakeClickHouseClient records the connection arguments and each command and query with its bound parameters. create, _create_collection, add_texts and _search now run their real code instead of being stubbed, so the tests check the generated SQL and parameters end to end.
  • test_elasticsearch_ja_vector.py imports the real modules and injects FakeElasticsearch through the real constructor by replacing the Elasticsearch class, so ping() and the version check run too. The fake index API records each creation, redis_client is a small recording fake, and the factory test uses a real Flask app context instead of a SimpleNamespace for current_app.
  • The factory tests compare the whole MyScale or Elasticsearch config instead of only the collection name.

Every test parameter and return is annotated. This changes test code only. Importing the real clickhouse_connect emits a DeprecationWarning from the library itself on Python 3.13 (the 'u' array type code).

Validation on Python 3.13:

  • All 18 cases pass, with the same test IDs and order as on main. The Elasticsearch JA tests also pass when test_elasticsearch_vector.py, which reloads these modules, runs first.
  • Focused pyrefly check with implicit-any-parameter, unannotated-return and implicit-any-empty-container set to error: 50 diagnostics on main under the unit-test configuration, 0 after under both the production and unit-test configurations. CI does not enable these three rules, so they were checked locally.
  • 17 deliberate source changes were applied one at a time, such as a dropped SET allow_experimental_object_type, a swapped connection user, a hard-coded LIMIT, a wrong sort order, a non-Japanese analyzer or a shorter cache TTL. The new tests fail for 16; the tests on main fail for 11, all of which the new tests also catch. The remaining change only affects documents with empty metadata, which these tests do not cover.
  • Ruff format/check and all six repository guards pass. The diff adds no MagicMock, cast, SimpleNamespace, spec, Any or type ignore.

Tests ran through the local Windows pytest wrapper; the lint and type-check commands were run individually rather than through make.

Screenshots

Before After
Focused Pyrefly diagnostics: 50 0
Seeded source changes caught: 11 of 17 16 of 17
Lines using MagicMock: 24 0

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've verified the change and added or updated tests where meaningful regression risk justifies coverage.
  • I've updated the documentation accordingly.
  • I ran make lint && make type-check (backend) and vp staged (frontend) to appease the lint gods

From Claude Code

…ding fakes

Both modules now import the real provider code, since clickhouse-connect and
elasticsearch are provider dependencies, and replace only the network client:
FakeClickHouseClient records each command and query with its parameters, and
FakeElasticsearch tracks created indices. Collection creation, inserts and
searches run their real code, the factory tests compare whole configs and
use a real Flask app context, and every test parameter and return is
annotated.
Copilot AI balanced review requested due to automatic review settings October 28, 2026 00:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@github-actions github-actions Bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Oct 28, 2026
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 80.30% 80.34% +0.05%
Strict coverage 79.98% 80.03% +0.05%
Typed symbols 77,347 77,454 +107
Untyped symbols 19,054 19,022 -32
Modules 3989 3989 0

@autofix-ci
autofix-ci Bot requested a review from a team October 28, 2026 00:54
@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.69421% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.54%. Comparing base (1f500e5) to head (41f233e).
⚠️ Report is 43 commits behind head on main.

Files with missing lines Patch % Lines
..._vdb_elasticsearch/test_elasticsearch_ja_vector.py 95.91% 4 Missing ⚠️
..._tests/src/dify_vdb_myscale/test_myscale_vector.py 97.22% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #44913      +/-   ##
==========================================
+ Coverage   89.52%   89.54%   +0.01%     
==========================================
  Files        6009     6012       +3     
  Lines      376843   377210     +367     
  Branches    76238    76278      +40     
==========================================
+ Hits       337380   337758     +378     
+ Misses      33465    33459       -6     
+ Partials     5998     5993       -5     
Flag Coverage Δ
api 89.41% <96.69%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants