add whisper base voice recognition (speech to text transcriptions) to capture audio playing in the site (questions asked by interviewer?) to transcribe and query gemma

[g-h-0-S-t]

add whisper base voice recognition (speech to text transcriptions) to capture audio playing in the site (questions asked by interviewer?) to transcribe and query gemma

maybe add a "read aloud" feature for responses too with play/pause/resume and stop button

[m13v]

interesting idea. the on-device whisper approach is the right move for this since you want low latency capture of what the interviewer is saying. a few things from building a similar pipeline: use chunked streaming (3-4 second windows with 0.5s overlap) so transcription is mostly done by the time someone stops talking. don't wait for the full utterance to finish before running inference. also whisper base is a good starting point but if you're running on Apple Silicon, the CoreML version of whisper small gives noticeably better accuracy for roughly the same latency as base on CPU. the read-aloud feature pairs well with AVSpeechSynthesizer, the premium voices on macOS are genuinely good now.

[m13v]

our transcription service that implements the chunked streaming pipeline with WhisperKit, CoreML model management, and the audio preprocessing I mentioned: https://github.com/m13v/fazm/blob/main/Desktop/Sources/TranscriptionService.swift

[kessler]

Thanks! I will look into it