ITEM-447-dynamic-thread-pool by fblackburn1 · Pull Request #187 · wazo-platform/xivo-lib-python

fblackburn1 · 2026-07-10T14:06:39Z

Why: cheroot spawns numthreads workers at startup and never resizes
the pool, so services permanently hold their peak thread count and
cannot absorb bursts. A monitor thread keeps a small buffer of idle
workers, growing and shrinking the pool between numthreads and max.

Design notes:

Growth covers the spare buffer plus the backlog idle workers cannot
absorb, capped at doubling per tick; shrink is one thread per tick.
Both bounds avoid grow/shrink oscillation on bursty load and thread
storms on unbounded pools (max_threads <= 0).
Workers are spawned directly instead of via pool.grow(): its
wait-until-ready spin can wedge the monitor forever, and its bulk
spawn leaks running workers when thread creation fails partway.
The monitor is non-daemon so grown workers match the initial ones
(the daemon flag is inherited); stop() joins it before stopping the
pool so an in-flight grow cannot race ThreadPool.stop().
Dead threads are culled before each measurement: cheroot culls only
inside shrink(), so pool stats would freeze after a shrink.
The private cheroot attributes used are probed once at startup and
scaling is disabled with a warning when one is missing, since the
cheroot version is not constrained in production.

Note

Medium Risk
Touches core HTTP serving and cheroot private APIs; mis-sizing or monitor races could affect concurrency and shutdown, though scaling is opt-in and guarded with fallbacks.

Overview
Adds DynamicWSGIServer on top of PatchedWSGIServer: a background monitor periodically resizes cheroot’s thread pool between numthreads and max, using minspare/maxspare and resize_interval. Sizing is driven by _compute_adjustment (grow for spare + backlog, capped per tick; shrink one thread at a time). Workers are added via _spawn_worker instead of pool.grow(); scaling turns off when cheroot internals are missing or min/max are invalid.

WSGIServer stays PatchedWSGIServer—callers opt into dynamic scaling explicitly. Coverage includes xivo/tests/test_wsgi.py and a dynamic-wsgi Docker integration test that asserts grow/shrink log lines under concurrent slow requests.

^{Reviewed by Cursor Bugbot for commit fa0fb63. Bugbot is set up for automated code reviews on this repo. Configure here.}

Why: cheroot spawns numthreads workers at startup and never resizes the pool, so services permanently hold their peak thread count and cannot absorb bursts. A monitor thread keeps a small buffer of idle workers, growing and shrinking the pool between numthreads and max. Design notes: - Growth covers the spare buffer plus the backlog idle workers cannot absorb, capped at doubling per tick; shrink is one thread per tick. Both bounds avoid grow/shrink oscillation on bursty load and thread storms on unbounded pools (max_threads <= 0). - Workers are spawned directly instead of via pool.grow(): its wait-until-ready spin can wedge the monitor forever, and its bulk spawn leaks running workers when thread creation fails partway. - The monitor is non-daemon so grown workers match the initial ones (the daemon flag is inherited); stop() joins it before stopping the pool so an in-flight grow cannot race ThreadPool.stop(). - Dead threads are culled before each measurement: cheroot culls only inside shrink(), so pool stats would freeze after a shrink. - The private cheroot attributes used are probed once at startup and scaling is disabled with a warning when one is missing, since the cheroot version is not constrained in production.

notion-workspace · 2026-07-10T14:06:44Z

Scale REST API worker threads dynamically with demand

wazo-community-zuul · 2026-07-10T14:15:52Z

Build succeeded.
https://zuul.wazo.community/zuul/t/local/buildset/f6303a67f735436fbfe847068d1f1ebd

✔️ wazo-tox-py311 SUCCESS in 2m 41s
✔️ tox-linters SUCCESS in 2m 33s
✔️ wazo-tox-integration-py311 SUCCESS in 6m 36s
✔️ debian-packaging-bookworm SUCCESS in 2m 28s

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit a8e08be. Configure here.}

wazo-community-zuul · 2026-07-16T13:10:45Z

Build succeeded.
https://zuul.wazo.community/zuul/t/local/buildset/2a98462d9c094aa088e2408ffb01b4d4

✔️ wazo-tox-py311 SUCCESS in 2m 43s
✔️ tox-linters SUCCESS in 2m 32s
✔️ wazo-tox-integration-py311 SUCCESS in 7m 00s
✔️ debian-packaging-bookworm SUCCESS in 2m 54s

DrPyser

Compared this against cheroot upstream's own attempt at the same feature — cherrypy/cheroot#256 ("Add a threadpool monitor that can resize the pool"), closed/unmerged. A maintainer's objection there was that this turns "a simple web-server into a load-balancer" and belongs in a wrapper, not in cheroot itself — which is exactly the approach taken here (subclass in the consuming project). Good call, and it explains why _REQUIRED_POOL_INTERNALS probing exists: there's no public API for this because upstream declined to add one.

Verified the private-attribute assumptions against the actually-pinned cheroot==9.0.0 (not cheroot HEAD, which has since added min/max validation this code predates): ThreadPool.__init__ in 9.0.0 has no max >= min check, so the 0 < pool.max < pool.min guard in __init__ is live code, not dead defensiveness; and pool.grow() really does leak orphaned threads on partial spawn failure in this version, which _grow_pool's one-at-a-time append correctly avoids. Growth/shrink math checked out against dynpool's (what #256 vendored) reference algorithm — matches on the backlog branch, and the doubling/one-per-tick caps are a deliberate, more conservative departure.

Three lower-priority items inline. Nothing here is a correctness bug in the growth/shrink behavior itself.

Why: prepare() never cleared the monitor stop flag, so after a cheroot stop/start cycle the new monitor thread exited on its first tick and pool scaling stayed disabled for the process lifetime. Also add unit tests for the monitor startup paths of prepare() and comments documenting two intentional tradeoffs raised in review (no dynpool-style floor-restore term, once-per-tick retry on grow failure).

wazo-community-zuul · 2026-07-24T16:02:55Z

Build succeeded.
https://zuul.wazo.community/zuul/t/local/buildset/971427b76cd945ca970a565b3f7a8039

✔️ wazo-tox-py311 SUCCESS in 2m 42s
✔️ tox-linters SUCCESS in 2m 28s
✔️ wazo-tox-integration-py311 SUCCESS in 6m 35s
✔️ debian-packaging-bookworm SUCCESS in 2m 54s

DrPyser · 2026-07-24T17:30:33Z

+                'min_threads (%s) > max_threads (%s): thread pool scaling disabled',
+                pool.min,
+                pool.max,
+            )


would add at least a debug log to allow sanity check of the effective parameters configured.

for sanity check of effective configuration

DrPyser

Tested through wazo-dird PR, works 💯 (quick exponential growth up to max + slow linear shrink).

wazo-community-zuul · 2026-07-24T17:56:12Z

Build succeeded.
https://zuul.wazo.community/zuul/t/local/buildset/59b359be8dce40348dc5d3fe55540bb4

✔️ wazo-tox-py311 SUCCESS in 2m 43s
✔️ tox-linters SUCCESS in 2m 33s
✔️ wazo-tox-integration-py311 SUCCESS in 6m 34s
✔️ debian-packaging-bookworm SUCCESS in 2m 18s

wazo-community-zuul · 2026-07-24T18:05:41Z

Build succeeded (gate pipeline).
https://zuul.wazo.community/zuul/t/local/buildset/3ef1258d77444808aec3e5a01a626894

✔️ wazo-tox-py311 SUCCESS in 2m 48s
✔️ tox-linters SUCCESS in 2m 34s
✔️ wazo-tox-integration-py311 SUCCESS in 6m 41s
✔️ debian-packaging-bookworm SUCCESS in 2m 24s

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-dao#350 Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes default concurrency and connection scaling (including raising default max_threads from 10 to 100), which can alter resource use and behavior under load on upgrade without explicit config overrides. Overview Switches wazo-agentd’s REST server from a fixed max_threads pool at startup to DynamicWSGIServer, which keeps rest_api.min_threads workers ready and scales up to max_threads under load. Defaults and docs now treat min_threads: 10 as the always-on baseline (threads and DB connections) and max_threads: 100 as the upper bound, replacing the previous model where max_threads alone fixed the pool size (formerly 10). Sample etc/wazo-agentd/config.yml and _DEFAULT_CONFIG in config.py are updated accordingly; 26.08 is noted in the changelog. Depends on xivo-lib-python (DynamicWSGIServer) and related xivo-dao work from linked PRs. Reviewed by Cursor Bugbot for commit d0c9a39. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: Charles <contact@charleslanglois.dev>

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes default concurrency and DB connection limits on upgrade (max_threads 10→100) and ties pool sizing to thread settings; misconfiguration could exhaust connections or under-provision under spike load. Overview REST concurrency no longer starts a fixed max_threads worker pool. The server switches to DynamicWSGIServer (xivo-lib-python) with min_threads threads always ready and max_threads as the upper bound under load. Configuration adds rest_api.min_threads (default 10) and raises the default max_threads from 10 to 100, with updated comments in etc/wazo-webhookd/config.yml and a 26.08 changelog entry. Database pooling is aligned with the new model: pool_size uses min_threads, and max_overflow scales toward max_threads in sync_db and SubscriptionService (the latter adds DB_POOL_SPARE_CONN extra overflow). RestApiConfigDict and integration test fixtures include min_threads. Reviewed by Cursor Bugbot for commit cd681a9. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Francois Blackburn Reviewed-by: Charles <contact@charleslanglois.dev>

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-dao#350 Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes request concurrency and associated DB connection usage at runtime; default max_threads rises from 10 to 100 on upgrade unless overridden. Overview Switches the REST API from a fixed thread pool to a dynamic one via DynamicWSGIServer (from xivo-lib-python). Startup keeps rest_api.min_threads workers ready; the pool grows under load up to rest_api.max_threads, which is now a ceiling instead of the count spawned at boot. Defaults and docs are updated: min_threads: 10, max_threads: 100 (previously only max_threads: 10 as a fixed size) in DEFAULT_CONFIG, etc/wazo-call-logd/config.yml, and CHANGELOG 26.08. http_server.py passes numthreads=min_threads and max=max_threads when creating the server. Reviewed by Cursor Bugbot for commit db27b09. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Francois Blackburn Reviewed-by: Charles <contact@charleslanglois.dev>

ITEM-447-dynamic-thread-pool Note: lib-python must be merged first to make linter pass ...(todo: update zuul to use depends-on for typing) Depends-on: wazo-platform/xivo-dao#350 Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes request concurrency and connection usage under load; behavior depends on DynamicWSGIServer from xivo-lib-python, so mis-tuned min/max could affect capacity or resource use. Overview The REST API WSGI layer switches from a fixed-size WSGIServer to DynamicWSGIServer, so worker threads (and associated DB connections) scale between a floor and a ceiling instead of spawning max_threads at startup. Configuration adds rest_api.min_threads (default 10, kept warm) and redefines max_threads as the upper bound (sample/default moves from 10 to 100). Typed defaults in config.py, shipped etc/wazo-amid/config.yml, and CHANGELOG 26.08 document the new semantics. Reviewed by Cursor Bugbot for commit 840fec1. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Francois Blackburn Reviewed-by: Charles <contact@charleslanglois.dev>

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-dao#350 Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes core request concurrency defaults and ties ARI connection pooling to max_threads, which can affect resource use under load; behavior relies on new xivo-lib-python WSGI support. Overview Replaces the fixed REST worker thread count with a dynamic pool between new rest_api.min_threads and rest_api.max_threads, wired through DynamicWSGIServer instead of WSGIServer. Defaults shift from a flat 10 threads to 10 ready at idle and up to 100 under load (config and sample config.yml updated; changelog 26.08). ARI HTTP connection pooling is aligned with peak concurrency: ari.connection.pool_size may be null and is then set to rest_api.max_threads at config load; explicit pool_size is unchanged. ARIClientProxy no longer has a built-in default pool size—tests and types reflect the new options. Risk note: Depends on companion changes in xivo-lib-python (DynamicWSGIServer) and related stack PRs. Reviewed by Cursor Bugbot for commit 1fc34a3. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Francois Blackburn Reviewed-by: Charles <contact@charleslanglois.dev>

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes how the CherryPy WSGI server scales concurrency for all REST traffic; mis-tuned min/max could affect latency or resource use under load, though request handling logic is unchanged. Overview Replaces a fixed REST worker thread count with a dynamic pool between rest_api.min_threads and rest_api.max_threads (via xivo’s DynamicWSGIServer). min_threads (default 10) is how many threads stay ready per HTTP/HTTPS listener; max_threads (default raised from 10 to 100) is now the upper bound under load, not the size spawned at startup. Defaults and sample etc/wazo-phoned/config.yml comments are updated accordingly; 26.08 changelog entry documents the new option and semantics. Reviewed by Cursor Bugbot for commit 8db785d. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: Charles <contact@charleslanglois.dev>

ITEM-447-dynamic-thread-pool Note: lib-python must be merged first to make linter pass ...(todo: update zuul to use depends-on for typing) Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes request concurrency and database pool sizing under load; mis-tuned min/max could affect capacity or connection usage, but no auth or data-model changes. Overview Introduces rest_api.min_threads so wazo-dird keeps a baseline of worker threads (and DB connections) ready, while max_threads becomes an upper bound the pool grows toward under load instead of a fixed count spawned at startup. The HTTP server switches from a fixed WSGIServer to DynamicWSGIServer (xivo-lib-python), starting with min_threads and capping at max_threads. init_db now uses pool_size=min_threads and a computed max_overflow so the SQLAlchemy pool can scale with demand (plus a small spare constant). Defaults and sample config.yml document the new option and raise the default max_threads to 100. Reviewed by Cursor Bugbot for commit 59d38b4. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Charles <contact@charleslanglois.dev> Reviewed-by: wazo-community-zuul[bot]

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-dao#350 Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes REST API concurrency and how many DB connections can be opened at peak; wrong min/max tuning could starve or overload the service, though defaults aim to reduce idle waste. Overview Switches wazo-confd’s REST server from a fixed thread pool sized by rest_api.max_threads to a dynamic pool via DynamicWSGIServer: min_threads (default 10) keeps that many worker threads (and DB connections) warm, while max_threads (default 100 in sample config, up from the old fixed 10) is only the upper bound under load. Defaults and docs are updated in wazo_confd/config.py, etc/wazo-confd/config.yml, and CHANGELOG 26.08. Actual pool behavior lives in dependent xivo-lib-python changes. Reviewed by Cursor Bugbot for commit 04f01ea. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Francois Blackburn Reviewed-by: Charles <contact@charleslanglois.dev> Reviewed-by: wazo-community-zuul[bot]

ITEM-447-dynamic-thread-pool Note: lib-python must be merged first to make linter pass ...(todo: update zuul to use depends-on for typing) Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Changes default concurrency and DB connection limits for all REST traffic; behavior depends on the merged xivo-lib-python DynamicWSGIServer implementation. Overview Replaces a fixed REST worker thread count with a dynamic pool that stays at rest_api.min_threads when idle and can grow up to rest_api.max_threads under load. Defaults move from a single max_threads: 10 to min_threads: 10 and max_threads: 100. The HTTP stack switches from WSGIServer to DynamicWSGIServer (xivo-lib-python), passing min_threads as the initial pool size and max_threads as the cap. SQLAlchemy is aligned via init_db: pool_size matches min_threads, and max_overflow is set from the thread ceiling plus a small spare (DB_POOL_SPARE_CONN) so connections can scale with demand instead of provisioning max_threads connections at startup. Config samples, built-in defaults, and the changelog document that max_threads is now a ceiling, not the number of threads always running. Reviewed by Cursor Bugbot for commit 2de4eea. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Francois Blackburn Reviewed-by: Charles <contact@charleslanglois.dev> Reviewed-by: wazo-community-zuul[bot]

ITEM-447-dynamic-thread-pool Depends-on: wazo-platform/xivo-lib-python#187 Why: max_threads was a fixed thread count spawned at startup, wasting idle threads and DB connections on quiet systems. The pool now grows and shrinks between the new min_threads setting and max_threads, which becomes a real ceiling. NoteMedium Risk Touches request concurrency and DB connection limits for all API traffic; mis-tuned min/max could exhaust PostgreSQL connections or cap throughput under load. Overview Replaces a fixed rest_api.max_threads worker count with a dynamic pool between new min_threads (25, always ready) and max_threads (200, load ceiling), so idle systems keep fewer threads and DB connections while spikes can grow. The REST stack switches from WSGIServer to DynamicWSGIServer (xivo-lib-python), passing numthreads=min_threads and max=max_threads; ReusePortWSGIServer now subclasses DynamicWSGIServer. init_db uses pool_size=min_threads and max_overflow sized to max_threads - min_threads plus a small spare (DB_POOL_SPARE_CONN), with defaults and etc/wazo-auth/config.yml updated accordingly. Tests mock DynamicWSGIServer instead of WSGIServer. Reviewed by Cursor Bugbot for commit a655f5a. Bugbot is set up for automated code reviews on this repo. Configure here. Reviewed-by: cursor[bot] Reviewed-by: Charles <contact@charleslanglois.dev> Reviewed-by: wazo-community-zuul[bot]

fblackburn1 added the mergeit label Jul 10, 2026

This was referenced Jul 10, 2026

ITEM-447-dynamic-thread-pool wazo-platform/wazo-confd#569

Merged

ITEM-447-dynamic-thread-pool wazo-platform/wazo-agentd#149

Merged

ITEM-447-dynamic-thread-pool wazo-platform/wazo-calld#412

Merged

feat(dynamic-threading): integration test

a8e08be

cursor Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread xivo/wsgi.py

DrPyser reviewed Jul 23, 2026

View reviewed changes

Comment thread xivo/wsgi.py

Comment thread xivo/wsgi.py

Comment thread xivo/wsgi.py

fblackburn1 requested a review from DrPyser July 24, 2026 15:56

DrPyser reviewed Jul 24, 2026

View reviewed changes

wsgi: info log for dynamic threadpool conf

fa0fb63

for sanity check of effective configuration

DrPyser approved these changes Jul 24, 2026

View reviewed changes

wazo-community-zuul Bot merged commit 0d62006 into master Jul 24, 2026
3 checks passed

wazo-community-zuul Bot deleted the ITEM-447-dynamic-thread-pool branch July 24, 2026 18:05

Uh oh!

Conversation

fblackburn1 commented Jul 10, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

notion-workspace Bot commented Jul 10, 2026

Uh oh!

wazo-community-zuul Bot commented Jul 10, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wazo-community-zuul Bot commented Jul 16, 2026

Uh oh!

DrPyser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wazo-community-zuul Bot commented Jul 24, 2026

Uh oh!

DrPyser Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

DrPyser left a comment

Choose a reason for hiding this comment

Uh oh!

wazo-community-zuul Bot commented Jul 24, 2026

Uh oh!

wazo-community-zuul Bot commented Jul 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fblackburn1 commented Jul 10, 2026 •

edited by cursor Bot

Loading