fix(embedder): route batch embedding to /v1/embeddings (Issue #629)#631
Open
jlin53882 wants to merge 3 commits intoCortexReach:masterfrom
Open
fix(embedder): route batch embedding to /v1/embeddings (Issue #629)#631jlin53882 wants to merge 3 commits intoCortexReach:masterfrom
jlin53882 wants to merge 3 commits intoCortexReach:masterfrom
Conversation
Fix Issue CortexReach#629: Ollama batch embedding fails after PR CortexReach#621 PR CortexReach#621 fixed single embedding by switching to /api/embeddings with prompt field. However, batch requests send payload.input as string[], which causes Ollama to reject with 'cannot unmarshal array into prompt'. Changes: - Route single input (string) to /api/embeddings + prompt (PR CortexReach#621) - Route batch input (string[]) to /v1/embeddings + input[] - Add response validation for count and non-empty embeddings - Add test/embedder-ollama-batch-routing.test.mjs with 5 test cases Testing: - Local testing with jina-v5-retrieval-test confirms both paths work - All 5 new tests pass Closes CortexReach#629
7b8a0f8 to
d2e68a6
Compare
Collaborator
|
Clean fix for the batch embedding regression introduced by #621. The root cause diagnosis is correct and the routing split is the right approach. Ready to merge with a few minor notes. Nice to have (non-blocking)
CI note: New test file Before merge: |
- Add embedder-ollama-batch-routing.test.mjs to CI manifest - Add comments explaining why provider options are omitted for Ollama batch - Add note about /v1/embeddings no-fallback assumption Reviewed by: rwmjhb
- Add embedder-ollama-batch-routing.test.mjs to CI manifest - Add comments explaining why provider options are omitted for Ollama batch - Add note about /v1/embeddings no-fallback assumption Reviewed by: rwmjhb
Contributor
Author
PR Review UpdatesAll review comments from @rwmjhb have been addressed: Fixed:
Rebase CheckVerified: Ready for merge! 🎉 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix Issue #629: Ollama batch embedding fails after PR #621
Problem
PR #621 fixed Ollama 0.20.5+ single embedding by switching to
/api/embeddingswithpromptfield. However, batch requests sendpayload.inputasstring[], causing Ollama to reject with:Solution
Route requests by input type inside
embedWithNativeFetch():string)/api/embeddingsprompt(PR #621 unchanged)string[])/v1/embeddingsinputAdditionally, add response validation to ensure batch responses contain the expected count and non-empty embeddings.
Changes
src/embedder.ts: Route based onArray.isArray(payload.input)test/embedder-ollama-batch-routing.test.mjs: 5 test cases covering routing and validationTesting
/api/embeddingswithprompt/v1/embeddingswithinput[]Limitation
This assumes
/v1/embeddingsbatch mode is valid for the target Ollama/model combination. Verified withjina-v5-retrieval-test.Closes #629