Skip to content

feat: add Microservices Load Shedding pattern (#3229) - #3599

Open
ylcn91 wants to merge 2 commits into
iluwatar:masterfrom
ylcn91:feat/microservices-load-shedding
Open

feat: add Microservices Load Shedding pattern (#3229)#3599
ylcn91 wants to merge 2 commits into
iluwatar:masterfrom
ylcn91:feat/microservices-load-shedding

Conversation

@ylcn91

@ylcn91 ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds the Microservices Load Shedding pattern as a new microservices-load-shedding module.

  • Problem: under a traffic spike or a slow dependency, requests pile up, latency grows for everyone and the service eventually falls over.
  • Solution: a LoadShedder performs capacity-based, priority-aware admission control at the service entry point. Excess requests are rejected immediately (fail fast) instead of being queued; low-priority work is shed first and a reserve of capacity is kept for critical requests.
  • Key components:
    • Priority, Request, Response: the request model with CRITICAL / NORMAL / LOW priorities.
    • LoadShedder: hard capacity, low-priority limit and critical reserve; lock-free compare-and-set admission so capacity is never exceeded; per-priority shed counters.
    • LoadShedException: the fast rejection ("503, retry later").
    • ShedGuardedService and RequestHandler: wrap the business logic and always release the capacity slot, even when the handler throws.
    • App: light load (everything admitted), payment provider slows down and orders pile up (LOW and NORMAL probes are shed, CRITICAL checkout still admitted), provider recovers (admission resumes), then a metrics summary. Logging traces every decision.
    • README.md: intent, real-world example, flowchart, code walkthrough, applicability, trade-offs, and how load shedding differs from rate limiting, throttling, backpressure and queue-based load leveling. PlantUML class diagram under etc/.
  • Tests: 13 JUnit 5 tests covering admission below capacity, priority-ordered shedding, the critical reserve, release, metrics, a 50-thread concurrency test proving capacity is never exceeded, slot release on handler failure, plus AppTest.
  • Module registered in the parent pom.xml. ./mvnw clean verify -pl microservices-load-shedding passes locally on JDK 21 and inside an eclipse-temurin:21 container.

Fixes #3229

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

PR Summary

Introduced a new Microservices Load Shedding pattern module providing capacity-based, priority-aware admission control at the service entry point. Excess requests are rejected immediately to protect latency for critical flows. Includes Priority, Request, and Response models, a LoadShedder with per-priority shedding, a ShedGuardedService wrapper, an App demo, unit tests, and README with diagrams, trade-offs and usage guidance.

Changes

File Summary
microservices-load-shedding/README.md Documentation detailing the Load Shedding pattern, its motivation, real-world context, an example Java implementation, class diagram, trade-offs, and how it relates to rate limiting, throttling, and backpressure.
microservices-load-shedding/etc/microservices-load-shedding.urm.png PNG diagram illustrating the LoadShedder and guarded service relationships per priority.
microservices-load-shedding/etc/microservices-load-shedding.urm.puml PlantUML class diagram capturing Priority, Request, Response, LoadShedder, ShedGuardedService, App and tests.
microservices-load-shedding/pom.xml Module pom.xml declaring dependencies, main class, and build settings for the new module.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java Demo application wiring the LoadShedder, ShedGuardedService, and a simulated payment provider to showcase phase-based load shedding.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java Exception type thrown on shedding; includes request id and priority for logging.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java Thread-safe admission controller with per-priority limits, lock-free in-flight updates, and shed counters.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java Enum defining CRITICAL, NORMAL, and LOW priorities for request admission.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java Immutable Request data object carrying id, priority, and description used by the shedder.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java Functional interface representing the business logic for admitted requests.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java Response model with ACCEPTED/REJECTED status and factory methods for admitted and shed results.
microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java Guarded service wrapper applying the shedder before delegating to business logic; ensures release on completion.
microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java Tests for App behavior and helper utilities; validates demo scenarios and provider behavior.
microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java Unit tests for LoadShedder covering shedding, capacity, release, and edge cases.
microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java Tests for ShedGuardedService behavior under admission, shedding and failure.
pom.xml Root pom.xml updated to include the new module in the build.

autogenerated by presubmit.ai

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
  • 50b5af8: feat: add Microservices Load Shedding pattern (#3229)
Files Processed (15)
  • microservices-load-shedding/README.md (1 hunk)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.puml (1 hunk)
  • microservices-load-shedding/pom.xml (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (2)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java [88-93]

    possible bug: "Logger field name mismatch with Lombok annotation"

  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java [68-74]

    possible bug: "Logger usage should use generated logger name (admission log)"

Skipped Comments (1)
  • pom.xml [263-263]

    enhancement: "Module wiring: add microservices-load-shedding"

@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Note on the automated review comments: LOGGER is the Lombok logger field name configured for this repository in lombok.config (lombok.log.fieldName = LOGGER), the same name every other module uses, so the code compiles as is. Local ./mvnw clean verify -pl microservices-load-shedding passes on JDK 21, also inside an eclipse-temurin:21 container.

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.87%. Comparing base (41625d8) to head (8ce89ea).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3599      +/-   ##
============================================
+ Coverage     83.79%   83.87%   +0.07%     
- Complexity     4277     4315      +38     
============================================
  Files          1121     1128       +7     
  Lines         15144    15285     +141     
  Branches        723      732       +9     
============================================
+ Hits          12690    12820     +130     
- Misses         2159     2169      +10     
- Partials        295      296       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ylcn91
ylcn91 force-pushed the feat/microservices-load-shedding branch from 50b5af8 to 6d9f7e0 Compare September 3, 2026 09:52
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up on the Codecov note: completed the LoadShedder constructor validation cases (every branch of the three checks plus a boundary configuration) and added AppTest cases for the demo helpers' interruption and failed-worker paths, which are now package-private for testing. 19 tests. Remaining uncovered lines are the demo's five-second safety timeouts and the CAS retry branch, which cannot be forced deterministically. ./mvnw clean verify -pl microservices-load-shedding passes locally on JDK 21 and in an eclipse-temurin:21 container.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (1)
  • 6d9f7e0: feat: add Microservices Load Shedding pattern (#3229)
Files Processed (15)
  • microservices-load-shedding/README.md (1 hunk)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.puml (1 hunk)
  • microservices-load-shedding/pom.xml (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (0)
Skipped Comments (2)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java [87-94]

    bug: "Fix logger usage with Lombok @slf4j in App"

  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java [65-75]

    bug: "Logger field name mismatch in ShedGuardedService"

@ylcn91
ylcn91 force-pushed the feat/microservices-load-shedding branch from 6d9f7e0 to c5ba691 Compare September 3, 2026 11:32
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Coverage follow-up: LoadShedder.acquire now uses AtomicInteger.getAndAccumulate with a pure accumulator instead of a hand-written compare-and-set loop, which is simpler and removes the untestable retry branch; the README snippet was updated to match. The demo helpers in App take a Duration so tests can exercise their timeout branches in milliseconds; main still uses the five-second value. JaCoCo now reports 100% instruction, branch and line coverage. Verified locally on JDK 21 and in an eclipse-temurin:21 container, five consecutive test runs, packaged jar runs end to end.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
  • c5ba691: feat: add Microservices Load Shedding pattern (#3229)
Files Processed (15)
  • microservices-load-shedding/README.md (1 hunk)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.puml (1 hunk)
  • microservices-load-shedding/pom.xml (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (2)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java [90-96]

    best_practice: "Logger name mismatch with Lombok @slf4j in App"

  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java [68-75]

    best_practice: "Logger usage mismatch with Lombok @slf4j in ShedGuardedService (admission log)"

Skipped Comments (0)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (3)
  • ba7abd5: docs: embed the rendered class diagram in the load shedding README

Render etc/microservices-load-shedding.urm.puml to PNG and embed it in the detailed explanation section, matching the other modules, instead of the inline mermaid block.

  • c23efa2: chore: report total shed count in the demo and add serialVersionUID

The demo summary now logs LoadShedder.getTotalShed() alongside the per-priority
counters, so the aggregate metric has a caller outside the tests. LoadShedException
declares serialVersionUID, matching the exception style used in the commander module.
The README sample output is refreshed from an actual run.

  • 9bac89a: fix: guard load shedder release and log the admitted in-flight count

  • LoadShedder.release() clamps the in-flight count at zero so an unmatched
    release cannot drive it negative and permanently inflate capacity

  • LoadShedder.acquire() returns the in-flight count including the admitted
    request, and ShedGuardedService logs that value instead of re-reading the
    counter, which under concurrency could report another request's number

  • README: replace the raw .puml link with an inline mermaid class diagram, and
    keep the code snippets in sync with the two changes above

Files Processed (8)
  • microservices-load-shedding/README.md (1 hunk)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.png (0 hunks)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.puml (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java (1 hunk)
Actionable Comments (0)
Skipped Comments (3)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java [90-96]

    maintainability: "Logger name mismatch with Lombok-generated logger"

  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java [68-69]

    maintainability: "Logger usage should match Lombok-provided logger"

  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java [71-78]

    maintainability: "Logger usage should match Lombok-provided logger (admission log)"

release() clamps the in-flight count at zero, acquire() returns the admitted count that the service logs, and the demo summary reports the total shed. The class diagram is a rendered PNG.
@ylcn91
ylcn91 force-pushed the feat/microservices-load-shedding branch from ba7abd5 to 8ce89ea Compare September 7, 2026 08:04

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
  • 8ce89ea: fix: harden load shedder release and report the admitted count

release() clamps the in-flight count at zero, acquire() returns the admitted count that the service logs, and the demo summary reports the total shed. The class diagram is a rendered PNG.

Files Processed (8)
  • microservices-load-shedding/README.md (1 hunk)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.png (0 hunks)
  • microservices-load-shedding/etc/microservices-load-shedding.urm.puml (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java (1 hunk)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java (1 hunk)
  • microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java (1 hunk)
Actionable Comments (1)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java [90-96]

    readability: "Logger name mismatch with Lombok @slf4j"

Skipped Comments (2)
  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java [61-83]

    readability: "Logger name mismatch with Lombok @slf4j (ShedGuardedService)"

  • microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java [191-193]

    readability: "Logger name mismatch in App.java report method"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Microservices Load Shedding pattern

1 participant