feat: implement voice mode improvements with continuous loop, audio error recovery, and TTS discovery

[devin-ai-integration]

Voice Mode Improvements

Summary

Implements the maple-voice-improvements spec with three main features:

1. Voice Mode — Continuous Loop State Machine

When a user starts recording on a TTS-capable platform (desktop/iOS), the app enters voice mode: a hands-free loop of Recording → Processing → Waiting → Generating → Playing → (500ms pause) → Recording. The mic button highlights when voice mode is active and acts as an exit button. Exit is also triggered on chat switch, new chat, or any overlay X button.

2. Audio Error Recovery

On transcription failure, the audio blob is retained in memory. The overlay shows an error state with the original recording duration, a Retry button (re-sends the same blob), and a Discard button. This works both inside and outside voice mode.

3. TTS Discovery Prompt & Enhanced Feedback

• One-time prompt: After the first successful voice message on TTS-capable platforms, if TTS models aren't installed, show an inline prompt ("Enable voice responses?") with a Download button (~264 MB) and dismiss (X).
• Generating animation: The speaker icon (TTSButton) pulses during TTS generation.
• Audio cues: Play mic-on.wav (gentle ascending tone) when recording starts and mic-off.wav (confirmation tone) after sending successfully.

Updates since last revision

TestFlight Bug Fixes (round 2)

• iOS audio cues still not playing (Bug 2a): AudioContext on iOS starts in "suspended" state. Added ctx.resume() before fetching and decoding the WAV file. Without this, source.start(0) silently fails on iOS because the context never transitions to "running".
• Previous recording duration flashes on re-entry (Bug 2b): setDuration(0) inside useEffect fires after the first render with stale state, causing a one-frame flash of the previous duration. Added a synchronous state reset during render (comparing prevEffectiveStateRef) that runs before React paints, eliminating the flash.
• Compact playback overlay layout imbalanced (Bug 2c): Waveform was pushed to the top and status text to the bottom due to gap-6 spacing. Reduced to gap-2 in compact mode (isCompact ? "gap-2" : "gap-6") for a tighter, more balanced layout.
• TTS auto-play after model download not triggering (Bug 2d): When ttsStatus transitions to "ready" mid-voice-loop, the user is already back in recording state with the mic active. The prevTtsStatusRef effect now stops the active recording (cleans up recorder and stream) before transitioning to "generating" state and calling speakAndWait.

Devin Review rounds 11–13

• Timer flash on re-entering recording (round 11): Added setDuration(0) at the start of the recording effect in RecordingOverlay.tsx to prevent a one-frame flash of the previous recording's stale duration before requestAnimationFrame resets it.
• Double mic-on audio cue (round 11): Removed playAudioCue("mic-on") from all 4 voice continuation caller sites in UnifiedChat.tsx. startRecording() is now the single source of truth for the mic-on cue, eliminating the stutter/echo caused by double playback.
• AudioContext resource leak (round 12): audioContextRef.current is now assigned immediately after new AudioContext() creation (before decodeAudioData, resume, etc.) so stopPlayback() can close it if any subsequent operation throws. Previously, the ref was set after several async operations, leaving orphaned contexts on failure.
• Empty preprocessed TTS text (round 12): speakInternal now throws Error("no_speakable_text") (instead of silently returning) when preprocessTextForTTS strips all content (e.g., code-only assistant responses). Both voice continuation catch blocks handle this by restarting recording with audio feedback, rather than silently activating the mic.
• Voice mode exit on startRecording failure (round 13): startRecording's catch block now calls exitVoiceMode() when voice mode is active, preventing a stuck "Recording" overlay when the microphone is unavailable (permission denied, device busy, etc.).
• Stale audioContextRef guard (round 13): The staleness check after audioContext.resume() now only nulls audioContextRef.current if it still points to this call's AudioContext, preventing a concurrent speakInternal call's ref from being clobbered.

TestFlight Bug Fixes (round 1)

• TTS auto-play after model download (Bug 1): Added a prevTtsStatusRef effect that watches for ttsStatus transitioning from non-ready → "ready" while voice mode is active. When the user downloads a TTS model mid-voice-loop, this retroactively speaks the last assistant message instead of silently skipping TTS.
• Compact overlay showing "Playing" state (Bug 2): The bottom input overlay (isCompact={true}) previously hid all status content and waveform — showing only a black overlay with an X button during TTS playback. Now shows status text (Playing, Generating, Waiting, Error) and animated waveform in compact mode for non-recording states.
• iOS audio cues (Bug 3): Replaced new Audio('/audio/file.wav') with Web Audio API (AudioContext + fetch + decodeAudioData + BufferSource). HTMLAudioElement.play() is unreliable in iOS Tauri WebView due to autoplay restrictions.
• Empty blob recovery race (Devin Review round 10): startRecordingRef.current() after empty blob recovery now uses setTimeout(0) to defer until React's setIsRecording(false) batch commits.
• Prettier formatting fix for CI.

Earlier fixes (rounds 1–9)

• Non-voice overlay dismissal (round 6): Added setVoiceState(null) after successful transcription in non-voice mode so the overlay dismisses.
• Voice mode error guard (round 6): Voice continuation effect now checks errorRef.current before proceeding with TTS. If handleSendMessage failed, voice mode exits gracefully instead of speaking stale messages.
• TTS failure loop break (round 7): speakInternal re-throws errors after handling them locally, allowing speakAndWait's .catch() to call exitVoiceMode().
• speak() wrapper error handling (round 8): The speak() wrapper (used by TTSButton) now catches errors from speakInternal to prevent unhandled promise rejections. Only speakAndWait (voice mode) propagates errors.
• exitVoiceMode stale TTS cleanup (round 5): Calls cancelTTSGeneration() and stopTTS() unconditionally instead of guarding on stale ttsIsGenerating/ttsIsPlaying.
• Earlier fixes (rounds 1–4): recordingStartTimeRef for accurate duration capture, startRecordingRef for stale closure in handleVoiceDiscard, handleSendMessageRef/handleTTSDiscoveryRef for stale closures, isTauri() guard on iOS TTS platform check, ?? for 0-second duration display, recording restart on empty blob, mic leak fix.

Review & Testing Checklist for Human

⚠️ Risk Level: YELLOW — Complex async state machine with 13 rounds of incremental Devin Review fixes + 4 TestFlight round 2 fixes; all iOS-specific bugs tested blind without device access. Key risks: (1) ctx.resume() audio cue fix is fire-and-forget with silent failure; (2) recorder cleanup in TTS auto-play effect happens in callback — potential race with speakAndWait; (3) synchronous setState during render (duration flash fix) could cause extra re-renders.

• TestFlight Bug 2a (iOS audio cues with ctx.resume()): On iOS, verify mic-on.wav plays when tapping mic button and mic-off.wav plays after successful send. If silent, check browser console for fetch/decodeAudioData errors.
• TestFlight Bug 2b (duration flash on re-entry): In voice mode, verify the timer shows 0:00 immediately when the overlay appears (no flash of previous duration).
• TestFlight Bug 2c (compact layout spacing): On iOS/mobile in voice mode during TTS playback, verify waveform and "Playing" text are vertically balanced (not pushed to edges).
• TestFlight Bug 2d (TTS auto-play after download): On iOS, start voice mode → record → dismiss TTS discovery (don't download) → record again → download TTS via discovery → verify last assistant message immediately shows "Generating audio" → "Playing" (no silent return to recording).
• Voice mode state machine: Test full loop (record → send → wait → TTS → auto-record) on desktop or iOS. Verify clean exit on: chat switch, new chat, mic button tap, X button, speaker icon tap.

Notes

• Code changes:
  • TTSContext.tsx: Added isGenerating, speakAndWait, cancelGeneration, generation sequence IDs, error re-throwing, no_speakable_text error, early audioContextRef assignment with stale guard
  • RecordingOverlay.tsx: Extended to 6 states with retry/discard, compact mode shows status/waveform for playback, synchronous duration reset during render (prevents flash)
  • UnifiedChat.tsx: Voice mode state machine, error recovery with blob retention, audio cues (Web Audio API with ctx.resume() for iOS), TTS discovery prompt, voice continuation effects, ref-based closures, exitVoiceMode() call in startRecording catch, no_speakable_text handling, TTS auto-play stops active recording before speaking
• Audio files: Added frontend/public/audio/mic-on.wav and mic-off.wav (both 394888 bytes, confirmed served by dev server)
• Ref pattern: startRecordingRef, handleSendMessageRef, handleTTSDiscoveryRef, recordingStartTimeRef, errorRef, prevTtsStatusRef, prevEffectiveStateRef to avoid stale closures
• Error propagation: speakInternal re-throws; speak() catches (TTSButton); speakAndWait propagates (voice mode)
• Web Audio API: AudioContext + ctx.resume() (iOS suspend fix) + fetch + decodeAudioData + BufferSource for iOS compatibility
• 15 rounds of fixes: 13 rounds Devin Review + 2 rounds TestFlight; tested blind on iOS TestFlight

---

Link to Devin Session: https://app.devin.ai/sessions/0c853f0e1ba84474971875a61f616769
Requested by: @marksftw

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[cloudflare-workers-and-pages]

Deploying maple with    Cloudflare Pages

Latest commit:	0cc9e9b
Status:	✅  Deploy successful!
Preview URL:	https://300bcef0.maple-ca8.pages.dev
Branch Preview URL:	https://devin-1772755465-voice-mode.maple-ca8.pages.dev

View logs

[marksftw]

@TestFlight build

[github-actions]

🚀 TestFlight deployment triggered! Check the Actions tab for progress.

[github-actions]

✅ TestFlight deployment completed successfully!

[devin-ai-integration]

Devin Review found 2 new potential issues.

View 24 additional findings in Devin Review.

[devin-ai-integration]

🟡 Stale voiceRetryCount in transcribeAndSend causes incorrect retry count on rapid retries

transcribeAndSend is a useCallback with voiceRetryCount in its dependency array (frontend/src/components/UnifiedChat.tsx:2073). In the catch block at line 2052, it reads voiceRetryCount from the closure and calculates newRetryCount = voiceRetryCount + 1. However, handleVoiceRetry (line 2138-2145) calls transcribeAndSend which may be holding a stale voiceRetryCount value. If the user taps Retry quickly twice, the second call uses the same closure as the first (since the setVoiceRetryCount from the first call hasn't triggered a re-render and new callback yet), so voiceRetryCount stays at the old value and the count never reaches 3 to show the extended error message. A ref should be used for voiceRetryCount to avoid the stale closure.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[marksftw]

@TestFlight build

[github-actions]

🚀 TestFlight deployment triggered! Check the Actions tab for progress.

[github-actions]

✅ TestFlight deployment completed successfully!

[marksftw]

@TestFlight build

[github-actions]

🚀 TestFlight deployment triggered! Check the Actions tab for progress.

[github-actions]

✅ TestFlight deployment completed successfully!