Claude/audio broadcast mvp fr p as#733
Claude/audio broadcast mvp fr p as#733braedonsaunders merged 2 commits intoclaude/build-openstudio-app-kgia5from
Conversation
…mode, transport hardening 6 workstreams implemented for production-ready audio broadcasting: WS1 - Robust Backing Track Broadcast (audio-engine.ts): - Idempotent createBroadcastStream() with safe teardown/rebuild - Auto-rebuild broadcast connections after sample rate changes - verifyBroadcastChain() diagnostics for node graph integrity - Guard in playBackingTrack/playStemmedTrack to rebuild if broadcastSongGain is null WS2 - Reliable State Sync Protocol (realtime.ts + useRoom.ts): - state:request/state:sync events replace fragile 500ms setTimeout - Full StateSyncPayload with queue, tempo, tracks, loops, songs, stems, permissions - Reconnection detection (SUBSCRIBED after CLOSED) with auto state request - Master failover re-broadcasts full state to all users - New joiners request state on connect with 3s retry fallback WS3 - Auto-Broadcast Store Changes (useRoom.ts + realtime.ts): - reliableBroadcast() wrapper: awaits result, retries once after 500ms - Critical broadcasts (play/pause/seek/tempo/state-sync) use reliable path - useTempoRealtimeBroadcast hook integrated for auto-broadcasting BPM/key/timesig - Stem toggle/volume broadcasts use reliable path WS4 - Native Bridge P2P Audio (Rust + TypeScript): - bridge.rs: Opus-encodes local audio and sends to P2P peers via NetworkManager - bridge.rs: Routes received P2P/relay audio to AudioEngine remote buffers - manager.rs: send_p2p_encoded_audio() and send_relay_audio() dual-path routing - p2p.rs: broadcast_audio_to_peers() for UDP audio transmission - peer.rs: Audio statistics tracking (packets, timestamps, track counts) - native-bridge.ts: P2P mode coordination (webrtc/p2p/hybrid auto-selection) - native-bridge.ts: Peer audio received callback and P2P stats events WS5 - Web Listener Experience (useAudioEngine.ts + useRoom.ts): - initializeListenerAudio() for receive-only initialization - Stable jitter buffer preset (100-200ms) for smooth listener playback - setListenerMode() for runtime performer/listener switching - No microphone permission request in listener mode WS6 - Transport Sync Hardening (realtime.ts + useRoom.ts): - Periodic position sync every 5s during playback (master → all) - Position drift correction when >200ms deviation detected - Seek debouncing (100ms) prevents broadcast flooding during scrub - Song position sync for multi-track timeline https://claude.ai/code/session_0164xqz1UmhSKFe8LgRDwKvq
TypeScript rejects direct cast from StemMixState (with specific keys) to Record<string, ...> since the index signature is missing. Cast through unknown as an intermediate step. https://claude.ai/code/session_0164xqz1UmhSKFe8LgRDwKvq
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
braedonsaunders
merged commit 7d2abeb
into
claude/build-openstudio-app-kgia5
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bad88d0da2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| setNetworkMode(mode: 'webrtc' | 'p2p' | 'relay' | 'hybrid'): void { | ||
| // Normalize 'relay' to 'webrtc' for internal tracking (relay uses the WebRTC path from our perspective) | ||
| this.currentNetworkMode = mode === 'relay' ? 'webrtc' : mode as 'webrtc' | 'p2p' | 'hybrid'; | ||
| this.send({ type: 'setNetworkMode', mode }); | ||
| } |
There was a problem hiding this comment.
evaluateAndSetNetworkMode can choose 'webrtc', but setNetworkMode forwards that value directly over the bridge transport. The native side only accepts p2p, relay, or hybrid for setNetworkMode (see native-bridge/src/protocol/server.rs), so this downgrade request is rejected while currentNetworkMode is still updated locally. In practice, when native peers leave, the JS layer can believe it switched back to WebRTC even though the native network stack stayed in the previous mode, causing inconsistent audio routing.
Useful? React with 👍 / 👎.
| // If track is playing, sync playback | ||
| if (payload.isPlaying && payload.currentTrack) { | ||
| const syncTime = Date.now() + 200; | ||
| loadBackingTrack(payload.currentTrack).then((success) => { |
There was a problem hiding this comment.
The new state:sync apply path only resumes playback when payload.currentTrack is present. For rooms currently playing a Song timeline (currentSongId without a queue track), joiners/reconnects will load metadata but never receive a play/position action, so they remain silent until someone manually triggers play again. This is a regression introduced by replacing late-join song rebroadcasts with the new full-state sync flow.
Useful? React with 👍 / 👎.
| this.seekDebounceTimer = setTimeout(() => { | ||
| this.seekDebounceTimer = null; | ||
| this.reliableBroadcast('track:seek', { trackId, timestamp, syncTime, userId: this.userId }); | ||
| }, 100); |
There was a problem hiding this comment.
broadcastSeek now delays sending by 100ms but forwards the original syncTime unchanged. Callers compute syncTime close to Date.now() + 100, so after debounce plus network transit, receivers commonly get a seek whose sync time is already in the past and apply it immediately, reducing cross-client timing alignment. Recomputing syncTime at send time (or skipping debounce for explicit seeks) avoids this drift.
Useful? React with 👍 / 👎.
2 participants
No description provided.