local whisper transcription for users without an elevenlabs key

[DonIsmaelito]

Lets people edit without an ElevenLabs account. Transcription is the first step of every session and used to hard-exit when ELEVENLABS_API_KEY was missing, even though the rest of the pipeline is ffmpeg and PIL.

Builds on the agent guidance PR; its commits show here until that PR merges.

What changes

• helpers/local_stt.py: local Whisper large-v3-turbo through mlx-whisper on Apple Silicon or faster-whisper (CTranslate2, int8 on CPU, float16 on CUDA) elsewhere. A hardware probe (local_stt.py probe) picks the library, reports install state, free disk, and model size, and prints the one install command, so a machine downloads only the model it will use. Weights come from huggingface_hub.snapshot_download pinned to a commit and land in the standard Hugging Face cache. A missing library exits with the exact install command; nothing here runs pip.
• helpers/transcribe.py and transcribe_batch.py: explicit engine selection, never a silent fallback. --engine wins, then VIDEO_USE_TRANSCRIBER (elevenlabs or local) from .env or the environment, then ElevenLabs when a key resolves. Every run prints engine: <name> (from <source>), and says when a key is present but unused. Transcripts gain a top-level engine key; files from before the key exist are treated as ElevenLabs and reused untouched. --force re-transcribes and swaps the file in only after the new run succeeds, so a paid Scribe transcript is never lost to a failed local run. The batch helper runs one worker for the local engine (one model per process).
• The local transcript keeps the shape every consumer reads (words with type, text, start, end, speaker_id), so pack_transcripts.py, render.py, and the EDL validators work unchanged.
• Decoder artifacts are guarded without a VAD dependency: segments Whisper marks as silent are dropped, words over 3 s are dropped, repetition loops are capped, and any echo of the filler-preserving prompt is removed. Whisper folds each pause into the start of the next word, so word starts are moved forward to the first audible energy frame (10 ms RMS frames, 20 % of the word's peak).
• One user-facing setting, VIDEO_USE_TRANSCRIBER. Which library runs is an implementation detail with developer-only --library and --model overrides. pyproject.toml adds two extras, stt-mlx and stt-cpu; core dependencies are unchanged.
• install.md step 5 asks once for a key or the local engine and persists the choice; SKILL.md gets a "Local transcription" subsection stating what the local engine does not provide (speaker labels, audio events, spacing entries; fillers best effort) and rewrites the two Whisper anti-pattern lines; README.md mentions the option.

Measured on an M2 MacBook Air, 8 GB, with the pinned fp16 turbo model

Two minutes of a keynote recording against the cached Scribe transcript for the same audio, comparing only words inside runs of three or more consecutive matches so a repeated phrase cannot pair with the wrong occurrence:

	onset median	onset p90	onset after a pause ≥ 0.4 s	word end median
raw Whisper word times	100 ms	560 ms	480 ms	40 ms
shipped (onset trim)	80 ms	400 ms	45 ms	40 ms

Remaining outliers are stutter runs ("the, the, the transformation") that Whisper collapses into one long word; Scribe keeps each repetition. That is the verbatim gap the docs now warn about. The 2 min sample transcribes in about 2.5 min on this machine after a one-time 1.6 GB download.

Tests

python -m pytest -q (57 tests). New coverage: library choice per machine, missing-library message, pinned download, word normalization, each artifact guard, onset trimming, verbatim prompt gating, dotenv precedence, the printed engine line, invalid values failing before any subprocess, the ElevenLabs path calling Scribe exactly once, an old transcript without engine being reused, and --force keeping the old file when the new run raises. No test loads a model or touches the network.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WF63xJ295MfvG1f2BTKQnx

---

Summary by cubic

Users without an ElevenLabs key can now transcribe and edit videos with a local Whisper engine. Transcription previously hard-exited when ELEVENLABS_API_KEY was missing, even though the rest of the pipeline is ffmpeg and PIL.

New Features

• Adds a local Whisper engine (mlx-whisper on Apple Silicon, faster-whisper elsewhere) with a hardware probe that prints the one CUDA-aware install command, downloads only the needed model, and preflights the environment before a run.
• Makes engine selection explicit: --engine wins, then VIDEO_USE_TRANSCRIBER, then the API key, and each run prints the engine and its source; quoted dotenv values are honored.
• Keeps the local transcript in the same word shape consumers read, so existing helpers and EDL validators work unchanged.
• Guards decoder artifacts without VAD: drops silent segments, words over 3 s, repetition loops (including non-Latin scripts), and prompt echoes, and trims word starts to the first audible energy frame using streamed onset detection.
• Adds a top-level engine key to transcripts; files without it are reused as ElevenLabs, and --force replaces a transcript only after success.

Migration

• Set VIDEO_USE_TRANSCRIBER=local in .env and install the stt-mlx, stt-cpu, or stt-cuda extra; install now asks once for a key or the local engine.
• The local engine lacks speaker labels, audio events, and spacing entries, and fillers are best effort.

^{Written for commit 961f3d4. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 19 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="helpers/timeline_view.py">

<violation number="1" location="helpers/timeline_view.py:59">
P3: This change forces the deprecated full-range `yuvj420p` pixel format, while every other ffmpeg call in the repo (`helpers/render.py` lines 124/310/692, `helpers/grade.py` line 292) uses `yuv420p`. `yuvj*` formats are deprecated in FFmpeg 7.0+ and produce a deprecation warning on every frame extraction (hidden here because stderr is DEVNULL), and may be removed in a future release. If the goal is full-range JPEG output to avoid washed-out frames, keep limited-range `yuv420p` and set the range explicitly instead, e.g. `scale=320:-2,format=yuv420p` plus `-color_range`/`out_range=full`, which is forward-compatible and consistent with the rest of the repo.</violation>
</file>

<file name="tests/test_comment_style.py">

<violation number="1" location="tests/test_comment_style.py:15">
P3: The punctuation guard doesn't enforce the convention the module docstring states. PUNCTUATION only matches [.,:;()\"'`], so a comment like "# why? wow!" passes audit() even though the rule says comments must not carry punctuation. Add the missing sentence punctuation (at least ? and !) to the character class.</violation>
</file>

<sub>Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: This change forces the deprecated full-range yuvj420p pixel format, while every other ffmpeg call in the repo (helpers/render.py lines 124/310/692, helpers/grade.py line 292) uses yuv420p. yuvj* formats are deprecated in FFmpeg 7.0+ and produce a deprecation warning on every frame extraction (hidden here because stderr is DEVNULL), and may be removed in a future release. If the goal is full-range JPEG output to avoid washed-out frames, keep limited-range yuv420p and set the range explicitly instead, e.g. scale=320:-2,format=yuv420p plus -color_range/out_range=full, which is forward-compatible and consistent with the rest of the repo.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At helpers/timeline_view.py, line 59:

<comment>This change forces the deprecated full-range `yuvj420p` pixel format, while every other ffmpeg call in the repo (`helpers/render.py` lines 124/310/692, `helpers/grade.py` line 292) uses `yuv420p`. `yuvj*` formats are deprecated in FFmpeg 7.0+ and produce a deprecation warning on every frame extraction (hidden here because stderr is DEVNULL), and may be removed in a future release. If the goal is full-range JPEG output to avoid washed-out frames, keep limited-range `yuv420p` and set the range explicitly instead, e.g. `scale=320:-2,format=yuv420p` plus `-color_range`/`out_range=full`, which is forward-compatible and consistent with the rest of the repo.</comment>

<file context>
@@ -54,7 +56,7 @@ def extract_frames(video: Path, start: float, end: float, n: int, dest_dir: Path
             "-frames:v", "1",
             "-q:v", "4",
-            "-vf", "scale=320:-2",
+            "-vf", "scale=320:-2,format=yuvj420p",
             str(out),
         ]
</file context>

Suggested change

	"-vf", "scale=320:-2,format=yuvj420p",
	"-vf", "scale=320:-2,format=yuv420p",

[cubic-dev-ai]

P3: The punctuation guard doesn't enforce the convention the module docstring states. PUNCTUATION only matches [.,:;()"'`], so a comment like "# why? wow!" passes audit() even though the rule says comments must not carry punctuation. Add the missing sentence punctuation (at least ? and !) to the character class.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At tests/test_comment_style.py, line 15:

<comment>The punctuation guard doesn't enforce the convention the module docstring states. PUNCTUATION only matches [.,:;()\"'`], so a comment like "# why? wow!" passes audit() even though the rule says comments must not carry punctuation. Add the missing sentence punctuation (at least ? and !) to the character class.</comment>

<file context>
@@ -0,0 +1,83 @@
+
+ROOT = Path(__file__).resolve().parents[1]
+SKIP_DIRS = {".venv", "venv", "node_modules", "__pycache__", ".git", "media", "edit"}
+PUNCTUATION = re.compile(r"[.,:;()\"'`]")
+DEFINITIONS = (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)
+
</file context>

Suggested change

	PUNCTUATION = re.compile(r"[.,:;()\"'`]")
	PUNCTUATION = re.compile(r"[.,:;()\"'`!?]")

[DonIsmaelito]

Addressed the review in 5da3820:

• local_stt.py: _plain keeps Unicode letters (P1), frame_energy streams fixed-size reads, validates mono 16-bit PCM, and is skipped when there are no words; preflight() settles and imports the library before any audio work; CUDA is used only when ctranslate2 can see a device, otherwise CPU with a note, and a stt-cuda extra adds the cuBLAS/cuDNN wheels.
• transcribe.py: dotenv_value unwraps quotes before looking for a comment; transcribe_one keeps the historical positional parameters and takes the engine controls keyword-only; the local preflight runs before extract_audio.
• transcribe_batch.py: preflight once before submitting work so a missing library exits with its install message instead of failing inside a worker.
• Docs: install step 5 also checks an exported VIDEO_USE_TRANSCRIBER, the synthetic check asks for confirmation before the model download and uses uv run python; Hard Rule 9 allows an explicit --force; README allows the free local check and tells manual installs to run the probe's command.

The timeline_view.py pixel format and the comment-style punctuation class are unchanged here; both come from the base branch in #145.

[cubic-dev-ai]

1 issue found across 9 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="helpers/transcribe_batch.py">

<violation number="1" location="helpers/transcribe_batch.py:118">
P3: The batch entry point preflights the local library once, but transcribe_one() preflights it again on every file it processes. Since the local engine is always a single worker, every local batch run re-runs the hardware probe and library import a second time. Drop the per-file preflight in transcribe_one (or guard it) so the "settle once" intent in batch mode actually holds.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P3: The batch entry point preflights the local library once, but transcribe_one() preflights it again on every file it processes. Since the local engine is always a single worker, every local batch run re-runs the hardware probe and library import a second time. Drop the per-file preflight in transcribe_one (or guard it) so the "settle once" intent in batch mode actually holds.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At helpers/transcribe_batch.py, line 118:

<comment>The batch entry point preflights the local library once, but transcribe_one() preflights it again on every file it processes. Since the local engine is always a single worker, every local batch run re-runs the hardware probe and library import a second time. Drop the per-file preflight in transcribe_one (or guard it) so the "settle once" intent in batch mode actually holds.</comment>

<file context>
@@ -111,6 +112,11 @@ def main() -> None:
+    # settle the local library once here so a missing install exits with its message instead of failing every worker
+    local_options = local_options_from(args)
+    if engine == "local":
+        preflight_local(local_options)
+
     workers = worker_count(engine, args.workers)
</file context>

[DonIsmaelito]

local_stt.preflight() is memoized per process (_PREFLIGHTED, added in 5da3820/961f3d4): the batch entry point pays for the probe and import once, and the per-file call inside transcribe_one is a dict lookup on an already-imported module. It stays there so direct callers of transcribe_one keep the fail-before-extraction guarantee. test_preflight_settles_once covers the single probe.

[DonIsmaelito]

Second pass addressed in the follow-up commit:

• require_library now receives the probe's CUDA state, so a missing faster-whisper on an NVIDIA host recommends stt-cuda.
• preflight() caches the settled library per process; transcribe_one keeps its own guarantee and the batch path no longer probes or imports twice.
• install.md's synthetic check uses a PY variable (uv run --no-sync python or python) so both install paths copy-paste correctly, and README's manual block tells the reader to use the extra the probe printed instead of hardcoding stt-mlx.