-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Labels
claudeSay, Pi for Claude AISay, Pi for Claude AIttsText to Speech (voice synthesis)Text to Speech (voice synthesis)
Description
When Claude launches one of its tool cards (e.g., computer use), SayPi’s TTS stream stops speaking a few words before the tool block appears. Once the tool completes, TTS resumes—but it jumps in mid-sentence rather than continuing seamlessly. The stop/start happens even though the final sentence before the tool is present in the DOM and is delivered to /speak/{uuid}/stream as a normal chunk.
Steps to Reproduce
- In Claude, prompt it to “show me the computer use tool” (or any request that triggers the computer/tool panel).
- Let the assistant respond while SayPi TTS is streaming.
- Listen as the narration approaches the tool preamble (e.g., “Sure! Here’s a quick demo. I’ll create a simple Python script and run it:”).
- Observe that audio stops before the tool card renders, then resumes only after the tool finishes (starting with the next sentence or bullet list).
Observed Behaviour
- Audio playback halts at “…I’ll create a simple Python script” (or similar) just before the tool card shows up.
- Once the tool completes, audio playback resumes where it left off with “…and run it:”, before proceeding to the next paragraph.
- Client logs confirm chunk
seq=3(47 chars, the full sentence) is sent before tool start, yet the audio for that chunk isn’t heard until after the tool run. - Example log snippet:
[saypi] Detected Claude tool use start …
[TTS KeepAlive] Starting keep-alive for f39f2389…
TextToSpeechService: Sending chunk seq=3 uuid=f39f2389… length=47
…
[saypi] Detected Claude tool use end …
TextToSpeechService: Sending chunk seq=4 uuid=f39f2389… length=323
Server side confirms chunk index 3 (“I’ll create a simple Python script and run it:”) is received at 10:16:09 and processed normally.
Expected Behaviour
TTS should continue speaking every chunk in order, right up to the tool card; it should not stop speaking mid-sentence before tool execution begins.
Notes
- Keep-alive behaviour looks normal (only a couple of PUTs from the client); the large number of “Keep-alive signal received” entries is server logging.
- Happens consistently when Claude renders any tool card, though the precise moment it pauses can move around, sometimes even stopping mid-word; not introduced by the recent keep-alive/indicator changes.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
claudeSay, Pi for Claude AISay, Pi for Claude AIttsText to Speech (voice synthesis)Text to Speech (voice synthesis)