Skip to content

Scope the history replication rate limiter - #12707

Open
jiechenz wants to merge 1 commit into
mainfrom
jiechenz/scoped-history-replication-rate-limiter
Open

jiechenz wants to merge 1 commit into
mainfrom
jiechenz/scoped-history-replication-rate-limiter

Conversation

@jiechenz

Copy link
Copy Markdown
Contributor

What changed?

history.EnableHistoryReplicationRateLimiter now takes {Enabled, Scope}:

  • all: rate limit every history append made by replication (same as the old bool, which is still accepted).
  • openForceReplication: only rate limit appends for force replication of running or paused workflows.

Why?

The rate limiter can only be turned on for all appends, and blocking live replication appends stalls the replication stream. Scoping it lets force replication of open workflows, which backfills full histories, be paced without touching live replication.

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

Potential risks

  • Waiting happens inside the replication task, so a low cap can push force replication tasks past the apply timeout and into the DLQ.
  • The setting is off by default and the OSS persistence rate limiter is a no-op, so nothing changes unless a deployment wires a limiter and enables it.

@jiechenz
jiechenz requested review from a team as code owners October 23, 2026 23:47
@jiechenz
jiechenz requested review from xwduan and yux0 October 23, 2026 23:47
history.EnableHistoryReplicationRateLimiter now takes {Enabled, Scope}. Scope "all"
rate limits every history append made by replication, as the bool did. Scope
"openForceReplication" only rate limits appends for force replication tasks of running
or paused workflows, so live replication never waits. A bool value is still accepted
and means "all". An invalid value, including a missing or unknown Scope, disables the
rate limiter.

The workflow state replicator records whether a task is force replication and the
workflow status in the context at ReplicateVersionedTransition and SyncWorkflowState,
and the three append sites check the scope against it. SyncState refetches keep the
force replication flag of the task that triggered them.
@jiechenz
jiechenz force-pushed the jiechenz/scoped-history-replication-rate-limiter branch from 0ed5cd5 to e579baf Compare October 24, 2026 00:00
@jiechenz jiechenz added the team/cgs-foundation Require cgs foundation team review label Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team/cgs-foundation Require cgs foundation team review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant