Scope
refactron/rag/indexer.py — add_chunks / index_repository
Problem
Each file path through _index_file → add_chunks calls self.embedding_model.encode(documents, ...) for only that file's chunks. SentenceTransformer/GPU throughput is much better with larger batches (e.g. hundreds of texts per call) than many small calls.
Suggested direction
- Accumulate
(chunks, documents) across files (with a max batch size / memory cap), then encode in batches and collection.add in corresponding batches.
- Optional: configurable
--batch-size for index CLI.
Acceptance
- Benchmark: indexing N files shows fewer
encode invocations and lower wall-clock time on representative repos (document rough numbers in PR).
Scope
refactron/rag/indexer.py—add_chunks/index_repositoryProblem
Each file path through
_index_file→add_chunkscallsself.embedding_model.encode(documents, ...)for only that file's chunks. SentenceTransformer/GPU throughput is much better with larger batches (e.g. hundreds of texts per call) than many small calls.Suggested direction
(chunks, documents)across files (with a max batch size / memory cap), thenencodein batches andcollection.addin corresponding batches.--batch-sizefor index CLI.Acceptance
encodeinvocations and lower wall-clock time on representative repos (document rough numbers in PR).