Conversation
Add openai_translate for non-English chat around RunPod; bind API keys as secrets, expose RunPod endpoint / model as params, and lazy-init Pinecone so deploy discovery does not need keys at import.
msobirovwayne
left a comment
msobirovwayne
left a comment
There was a problem hiding this comment.
reviewed ayaan's translation code
| kwargs: dict[str, Any] = { | ||
| "model": model, | ||
| "messages": messages, | ||
| "temperature": 0.2, |
There was a problem hiding this comment.
For translation tasks, 0 might be better for temperature because we want deterministic literal output. 0.2 is low but could still introduce randomness.
| continue | ||
| by_i[int(x["i"])] = str(x.get("t", "")) | ||
|
|
||
| if len(by_i) != len(payloads): |
There was a problem hiding this comment.
The validation checks that the count matches, but doesn't verify that every expected index i (0 through last_i) is actually present — by_i could have the right length but with wrong/duplicate keys. Consider checking set(by_i.keys()) == set(range(len(payloads))) instead
| return s in ("english", "en", "") | ||
|
|
||
|
|
||
| def _client() -> OpenAI: |
There was a problem hiding this comment.
This creates a brand new client object each time normalize_to_english and translate_english_to_ui_language are called within the same request. Consider caching it as a module-level singleton (like the Pinecone client pattern used in main.py) to avoid redundant initialization overhead.
| if json_object: | ||
| kwargs["response_format"] = {"type": "json_object"} | ||
| try: | ||
| resp = client.chat.completions.create(**kwargs, max_completion_tokens=8192) |
There was a problem hiding this comment.
Using a bare except TypeError to detect the API version differences is fragile. Might be better to check the SDK verrsion explicitly or pin the SDK version in requirements.txt
2 participants
Add openai_translate for non-English chat around RunPod; bind API keys as secrets, expose RunPod endpoint / model as params, and lazy-init Pinecone so deploy discovery does not need keys at import.