Pinecone upsert optimization #10
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ivanazeljkovic
changed the title
ivanazeljkovic
changed the title
* remove the preview package * more cleanup * fix linter
* fix: load_from_deepset_cloud without name in yaml * fix tests * add reno
* added pods and pod_type arg to PineconeDocumentStore * added a test * added releasenotes * added pod, pod_type to mock pinecone index and releated methods * removing test file from others * added integration test * fixes * set default values for new parameters * rm test that needs a pro plan --------- Co-authored-by: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
…set-ai#6529) * cast to PromptModelInvocationLayer * fix pylint pointless-exception-statement * use two variables to avoid re-assignment * black * use mocked tokenizer in unit test
…et-ai#6533) * convert use_auth_token into token * add explanatory comment
* remove OpenAIAnswerGenerator * reno * Remove BaseGenerator and GenerativeQAPipeline * remove more references * docs * leftover * reno
* Add MongoDB Atlas document store * Add tests * Fix linting * Add release notes + remove extra roman import * Add pymongo driver info * mypy and pylint fixes * Use future import annotations * fix errors in tests * remove from __future__ import * Fix * remove tests * add docstring --------- Co-authored-by: Julian Risch <julian.risch@deepset.ai>
…set-ai#6547) * Fix minor_version_release.yml to run on the v1.x branch * review
* feat: add bedrock embeddings to embedding encoder * reno & black * feat: refactoring for bedrock embedding encoder * feat: bedrock embedding encoder * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring, add cohere * feat: bedrock refactoring, add cohere * pylint: disable too-many-return-statements in method * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * feat: bedrock refactoring * fix mypy and pylint errors * manually run precommit * refactor init * fix cohere truncate and refactor embed * fix mypy * proper exception handing --------- Co-authored-by: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Co-authored-by: anakin87 <stefanofiorucci@gmail.com> Co-authored-by: tstadel <thomas.stadelmann@deepset.ai>
…ai#6574) * Changed standard final_answer_pattern in base.py to avoid stopping match with newline * added testing on multiline * added testing on final_answer_pattern for multiline matching * reformatting * added release note
…eciprocal rank fusion (deepset-ai#5704) * Change type of PromptModel invocation_layer_class init param (deepset-ai#6497) * fix: mypy `"str" not callable` for `PromptModelInvocationLayer` (deepset-ai#6529) * cast to PromptModelInvocationLayer * fix pylint pointless-exception-statement * use two variables to avoid re-assignment * black * use mocked tokenizer in unit test * Add normalization and weighting for `JoinDocuments` reciprocal rank fusion * Add weights and score normalization for reciprocal rank fusion in JoinDocuments node. * Fix black-jupyter * Fix JoinDocuments test for rrf + score normalization --------- Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com> Co-authored-by: Julian Risch <julian.risch@deepset.ai>
* Add model_kwargs to SentenceTransformersRanker * Add unit test * Add unit tests for the torch dtype extraction * Add release notes * Fix formatting * Fix patch * Make function more explicit
ivanazeljkovic
force-pushed
the
feature/pinecone-upsert-optimization
branch
from
5918ded to
c25c4e9
Compare
ivanazeljkovic
force-pushed
the
feature/pinecone-upsert-optimization
branch
from
853e88d to
5918ded
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pinecone upsert optimizations include few changes:
NOTE: Please do not merge this pull request