Add frame processor support for audio streams #533
Conversation
| T = TypeVar("T", bound=Union[AudioFrame, VideoFrame]) | ||
|
|
||
|
|
||
| class FrameProcessor(Generic[T], ABC): |
There was a problem hiding this comment.
suggestion (non-blocking): This might be over abstracting, but we could make this more generalizable by defining an AuthenticatedFrameProcessor interface which inherits from a more general FrameProcessor one.
There was a problem hiding this comment.
true, we can always go the other way around later with an Unauthenticated... (not that name, but something better) that stubs out the setCredentials call.
| num_channels: int = 1, | ||
| frame_size_ms: int | None = None, | ||
| noise_cancellation: Optional[NoiseCancellationOptions] = None, | ||
| noise_cancellation: Optional[NoiseCancellationOptions | FrameProcessor[AudioFrame]] = None, |
There was a problem hiding this comment.
Should we have another field name than noise_cancellation?
Like just processors?
| def _update_stream_info( | ||
| self, | ||
| *, | ||
| room_name: str, | ||
| participant_identity: str, | ||
| publication_sid: str, | ||
| ): ... | ||
|
|
||
| @abstractmethod | ||
| def _update_credentials(self, *, token: str, url: str): ... | ||
|
|
||
| @abstractmethod | ||
| def _process(self, frame: T) -> T: ... | ||
|
|
||
| @abstractmethod | ||
| def _close(self): ... |
There was a problem hiding this comment.
Do we need those methods (e.g: _update_credentials) ? Seems like an implementation detail?
There was a problem hiding this comment.
I'm thinking this interface could be used by anybody, more like a general purpose API
There was a problem hiding this comment.
I think the underlying Rust implementation would need to make calls to _update_credentials when tokens have refreshed.. this is the same thread as jacob's comment
| owned_buffer_info = audio_event.frame_received.frame | ||
| frame = AudioFrame._from_owned_info(owned_buffer_info) | ||
| if self._processor is not None: | ||
| frame = self._processor._process(frame) |
No description provided.