Skip to content

fix(ui): gate voice orb on a true ready signal, not start() returning (#38)#47

Open
nithiink wants to merge 2 commits into
mainfrom
fix/orb-ready-gate-v2
Open

fix(ui): gate voice orb on a true ready signal, not start() returning (#38)#47
nithiink wants to merge 2 commits into
mainfrom
fix/orb-ready-gate-v2

Conversation

@nithiink

Copy link
Copy Markdown
Owner

Fixes #38.

Problem

The voice orb's --amp mic-volume animation began scaling from live mic input before the voice connection was actually ready, across all three providers (Gemini, OpenAI, Azure). The orb gate was keyed on connectedRef, which mirrors connected — and connected flips true the instant s.start() resolves. But start() returns before each provider's post-start application handshake completes, while the mic analyser is already attached. So the moment connected flipped, the orb twitched from mic input during the still-in-progress handshake.

  • Gemini: start()/openSocket() return before setupComplete arrives.
  • OpenAI/Azure: start() resolves right after setRemoteDescription, but the data channel openconfigureSession() fires asynchronously after that. The old code also emitted state: "listening" prematurely at the end of start().

Fix

Introduce an explicit ready transport event emitted at each provider's genuine ready point, and gate the orb on it instead of on start() returning.

  • lib/voice.ts — add { type: "ready" } to RealtimeEvent.
  • lib/gemini.ts — emit ready on setupComplete.
  • lib/realtime.ts — emit ready/listening from the data-channel open handler (after configureSession()), and remove the premature signals at the end of start().
  • components/VoiceAgent.tsx — gate orbLoop on a readyRef driven by the ready event instead of connectedRef; reset it in connect()/disconnect().

Comment cleanup

Per review, trimmed the inline comments to remove redundancy: the orb-gate rationale now lives once in lib/orb.ts (its home) and on the ready event in voice.ts; the duplicate restatements in orbLoop, onEvent, and the readyRef declaration were removed or shortened to one-liners.

Tests

Extracted the orb gate/amplitude math into pure helpers (lib/orb.ts: rmsAmp, orbTarget, envelopeStep) so the gate timing is unit-testable, and added lib/orb.test.ts. The core case asserts --amp stays 0 while the gate is closed even under loud mic input, plus envelope decay/rise behavior.

Checks run before opening: node --test (12/12 pass) and tsc --noEmit (clean).

🤖 Generated with Claude Code

nithiink and others added 2 commits June 16, 2026 16:08
…#38)

The orb's --amp mic-volume animation scaled from live mic input before the
voice connection was actually ready, for all three providers. The gate was
keyed on `connected`, which flips true the instant `s.start()` resolves — but
`start()` returns before each provider's post-start handshake completes, while
the mic analyser is already attached. So the orb twitched from mic input during
the still-in-progress handshake.

Fix: introduce an explicit `ready` transport event emitted at each provider's
genuine ready point, and gate the orb on it.

- voice.ts: add `{ type: "ready" }` to RealtimeEvent.
- gemini.ts: emit `ready` on `setupComplete` (not when openSocket() returns).
- realtime.ts: emit `ready`/`listening` from the data-channel `open` handler
  after configureSession(), and drop the premature signals at the end of
  start() (which fired before the channel was even open).
- VoiceAgent.tsx: gate orbLoop on a `readyRef` driven by the `ready` event
  instead of `connectedRef`; reset it in connect()/disconnect().

Extract the orb gate/amplitude math into pure helpers (lib/orb.ts) so the gate
timing is unit-testable, and add lib/orb.test.ts — the core case asserts --amp
stays 0 while the gate is closed even under loud mic input.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code review surfaced that readyRef was only reset on connect()/disconnect(),
so during a Gemini reconnect (setupComplete → socket drop → reconnect) the orb
kept scaling from live mic input across the outage — the same class of bug this
PR fixes. The transport already emits `state: "connecting"` from its reconnect
path, so close the gate on that and let the next `ready` (on the fresh
setupComplete) re-open it. The orb now rests through the reconnect window.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Voice orb scales from mic input before connection is ready (all providers)

1 participant