feat(app): add managed local inference thread control with measured benchmarks

[parthashirolkar]

Managed Local currently fixes inference threads automatically, so users cannot choose how much CPU capacity to give local transcription. This adds a Settings slider bounded by the device's available logical processors, a restore-default action, and guidance about the CPU/latency tradeoff. Changes reload the recognizer on next use, including when settings change during startup, while preserving active inference.

Config adds nullable managedLocalThreads; existing installations retain the current automatic policy. Main passes the validated count through the existing model-load message to the CPU utility process. No capture behavior, model export, or default ONNX Runtime tuning changes are included.

The research report records 578 measured native decodes and the limitations of the single-device screening results. It is an MDX document with an embedded interactive Chart.js scatter plot, clip filters, latency/speedup views, and Cursor preview instructions. Chart.js is a pinned development dependency used by the documentation.

Validation:

• 44 focused config/runtime tests and 48 Settings tests passed, including failed-reset retry and startup thread changes.
• App TypeScript and focused ESLint passed.
• MDX compilation/server rendering preserved all four tables; ten chart filter/metric combinations and unmount cleanup passed with local Chart.js.
• Real Electron utility-process transcription at 1 and 2 threads passed; the user verified the Settings UI in the native app.
• No repository-wide or packaging checks were run locally; CI owns the full suite.

Created by GPT-6.1-Sol through the Codex harness in T3 Code.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by CodeRabbit

• New Features
  • Managed Local settings now include a CPU inference-thread slider and a Use default option. The slider range reflects the available processor capacity.
• Improvements
  • Settings provide guidance on CPU usage and speed. Thread-count changes take effect on the next transcription without interrupting an active transcription or model load.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 44 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: c9b62502-1f28-4a6d-87b6-7b89177b53b7

📥 Commits

Reviewing files that changed from the base of the PR and between 16936a3 and ac77cf2.

📒 Files selected for processing (2)

• app/src/main/__tests__/managed-local-runtime.test.ts
• app/src/main/managed-local-runtime.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: c10cbbd3-3688-4c88-8015-8448dd7c9b48

📥 Commits

Reviewing files that changed from the base of the PR and between b8b1fb6 and 16936a3.

⛔ Files ignored due to path filters (1)

• bun.lock is excluded by !**/*.lock

📒 Files selected for processing (17)

• app/CHANGELOG.md
• app/CONTEXT.md
• app/docs/research/parakeet-quality-and-cpu.mdx
• app/package.json
• app/scripts/parakeet-experiment.ts
• app/src/main/__tests__/config.test.ts
• app/src/main/__tests__/managed-local-runtime.test.ts
• app/src/main/config.ts
• app/src/main/index.ts
• app/src/main/local-stt.ts
• app/src/main/managed-local-runtime.ts
• app/src/main/managed-local-threads.ts
• app/src/main/managed-local.ts
• app/src/renderer/settings/TranscriptionSettings.tsx
• app/src/renderer/settings/__tests__/SettingsWindow.test.tsx
• app/src/types/ipc.ts
• app/src/types/sherpa-onnx-node.d.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Adds a Managed Local inference-thread setting with processor-bounded values and runtime reload behavior. Also adds a Parakeet experiment runner and a research report with measured results and an interactive thread-count chart.

Changes

Managed Local thread setting

Layer / File(s)	Summary
Thread setting and limits contract app/src/types/ipc.ts, app/src/main/managed-local-threads.ts, app/src/main/config.ts, app/src/main/__tests__/config.test.ts, app/src/main/index.ts	Adds the nullable thread setting, its validation and default, persistence checks, and inference limits in the managed-local model snapshot.
Thread loading and runtime replacement app/src/main/managed-local.ts, app/src/main/managed-local-runtime.ts, app/src/main/local-stt.ts, app/src/main/__tests__/managed-local-runtime.test.ts	Passes numThreads to the utility process. A changed count uses a replacement process on a later transcription, without interrupting active inference.
Settings control and documentation app/src/renderer/settings/TranscriptionSettings.tsx, app/src/renderer/settings/__tests__/SettingsWindow.test.tsx, app/CHANGELOG.md, app/CONTEXT.md	Adds the inference-thread slider and default action, tests save and error behavior, and documents when changes take effect.

Parakeet benchmark tooling and report

Layer / File(s)	Summary
Experiment runner and measurements app/scripts/parakeet-experiment.ts, app/src/types/sherpa-onnx-node.d.ts	Adds audio helpers, quality and speed experiment suites, model and decode measurements, and JSON output. Updates the Sherpa ONNX declarations used by the runner.
Results report and interactive chart app/docs/research/parakeet-quality-and-cpu.mdx, app/package.json	Adds the experiment findings, reproduction instructions, an interactive thread-count chart, and the chart.js development dependency.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant SettingsUI
  participant MainConfig
  participant ManagedLocalTranscriber
  participant UtilityProcess
  SettingsUI->>MainConfig: Save managedLocalThreads
  ManagedLocalTranscriber->>MainConfig: Read managedLocalThreads
  ManagedLocalTranscriber->>UtilityProcess: Load model with numThreads
  UtilityProcess-->>ManagedLocalTranscriber: Report ready

Loading

Merge Risk: ⚪ Minimal · up to 16936

The thread controls preserve automatic defaults and apply changes without interrupting active inference. No merge-blocking issue is established; merge after normal CI checks pass.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 16936

This is a bounded, device-local resource-control change that preserves existing defaults. No new external access was identified, but uncommon interruption paths and release behavior are not fully established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The added resource authority is bounded to local inference CPU allocation, increasing the selectable ceiling from the previous automatic cap to available parallelism. Its direct availability impact is local contention; the inspected field does not confer additional credential, filesystem, or network authority.

Trust Boundaries and Controls

• observed — The main setter rejects non-null counts outside the integer range from one through available parallelism. Persisted invalid values normalize to null, and the utility receiver repeats the shared validation before native recognizer creation.
• observed — The existing preload bridge exposes generic IPC invocation, and config:set does not inspect sender identity. Base-to-head comparison shows that this authorization limitation predates the PR; the new setting uses that existing channel rather than adding an external entrypoint.

Resilience and Maintainability Implications

• observed — Base and head share an exceptional-startup catch that clears readiness without explicitly disposing an already assigned child. Settings-driven replacement reuses this startup path. Complete containment of synchronous process-setup failures and overlapping shutdown remains unproven, rather than an established new security finding.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 13 files. (4 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: Managed Local inference thread control. It also accurately mentions the measured benchmarks included in the pull request.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 13 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Confidence Score: 4/5

[Medium risk] Adds inference thread control to local speech recognition.

The PR is not ready to merge because a thread change can miss the next transcription while another is running.

Findings

1.  Next dictation uses old threads ▶

Summary

Managed Local now lets people choose how many CPU threads local transcription uses, while keeping the existing automatic setting as the default. The PR also adds a Parakeet research report and a runner for exploring quality and speed measurements.

• Adds a Settings slider, a restore-default action, and guidance about CPU use and speed.
• Passes the chosen count to the local recognizer and applies changes on a later use.
• Adds an interactive benchmark chart and tooling for measuring audio variants and thread counts.

_{Reviews (1) · Last reviewed commit: "fix(app): harden inference thread contro..."}