Fix Image.frombytes() mode param usage #1365
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
currently we're passing
frame.format
which typically contains file format strings like 'PNG' or 'JPEG' as the mode parameter to PIL.Image.frombytes()This function actually expects image modes like 'RGB', 'RGBA', etc
I standardized all Image.frombytes() calls to use 'RGB' mode instead, fixed this in six files:
src/pipecat/transports/base_output.py
src/pipecat/services/gemini_multimodal_live/events.py
src/pipecat/services/moondream.py
src/pipecat/services/google/google.py
src/pipecat/services/anthropic.py
src/pipecat/processors/aggregators/openai_llm_context.py
For most of the places where it's used, 'RGB' as the image mode works perfectly since it is immediately convert to JPEG anyway. However in
base_output.py
, I went with 'RGB' for consistency, though I wonder if preserving other modes is useful? Not sure if a png with an alpha channel can be passed here for example, and we might prefer RGBA in that case (to account for that, we might need to add a dedicated class member for storing image mode inImageRawFrame
perhaps)