Skip to content

fix(redis): make reshard failures and target auth explicit - #552

Open
weicao wants to merge 2 commits into
mainfrom
fix/redis-reshard-server-errors
Open

fix(redis): make reshard failures and target auth explicit#552
weicao wants to merge 2 commits into
mainfrom
fix/redis-reshard-server-errors

Conversation

@weicao

@weicao weicao commented Jul 25, 2026

Copy link
Copy Markdown

What changed

  • propagate Redis reshard failures instead of unwrapping/discarding them
  • call Value::extract_error() for every CLUSTER SETSLOT and MIGRATE reply
  • require OK for SETSLOT while permitting Redis' legitimate MIGRATE NOKEY outcome
  • derive destination MIGRATE credentials from the same merged target URL and redis-rs connection contract
  • send AUTH for password-only targets and AUTH2 for username/password targets
  • retry password-only AUTH only for the two exact Redis <6 AUTH2 incompatibility errors; auth failures and near-matches fail closed
  • preserve the original move semantics: no COPY or REPLACE
  • before returning success, use one 30-second wall-clock deadline to verify every original master reports:
    • the same original master set
    • no migrating/importing markers
    • complete, unique ownership of slots 0..16383
    • a balanced distribution (max - min <= 1)
    • an identical slot-owner map
  • add the required Unreleased changelog entry

Why

The first runtime defect was a false success: a FalkorDB 4.14.12 reshard test using ape-dts image tag 2.0.26-alpha.16 produced two fresh runs where the KubeBlocks OpsRequest reached Succeed, but 29/31 slots retained migrating/importing markers and associated data was unreadable; one isolated run completed cleanly.

The extractor used redis-rs 0.27.6 ConnectionLike::req_packed_command, which returns Redis server -ERR replies as Ok(Value::ServerError(...)). The previous code discarded those values, so a failed SETSLOT/MIGRATE could still let the action exit zero. A dirty retry could also have an empty move plan and return success without proving terminal slot ownership.

A fresh exact-head fixcheck then exposed the second defect: with destination authentication enabled, the OpsRequest correctly failed and ape-dts exited 101 with Target instance replied with error: NOAUTH Authentication required. The destination connection itself used merged credentials, but MIGRATE omitted its target-side AUTH/AUTH2 options.

This evidence implicates the ape-dts reshard path; it does not claim a regression in the addon change that exposed the runtime case.

Verification

  • cargo test -p dt-connector redis_reshard_extractor::tests: 20 passed
  • cargo test -p dt-connector: 133 passed, 0 failed, 1 ignored
  • cargo fmt --all -- --check: passed
  • git diff --check: passed
  • cargo clippy -p dt-connector --all-targets: exit 0; existing warnings are outside the modified file
  • independent read-only design-contract review session 019f9a1d-651b-73f0-8e2b-043db7546f73: VERDICT: NO BLOCKER

Residual nonclaims: a later mutation failure is propagated but earlier partial mutations are not automatically rolled back; the new 30-second bound applies to terminal async verification, not the pre-existing synchronous discovery/mutation I/O; the stalled-I/O test uses a controlled future rather than a live network stall; the Redis <6 fallback is limited to exact known incompatibility replies and is not a broad retry policy. Fresh runtime acceptance of the authenticated fix remains pending.

@weicao weicao changed the title fix(redis): propagate reshard command failures fix(redis): make reshard failures and target auth explicit Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant