Conversation
- Move retry, throttle, and fallback coordination logic from LlmClient into new LlmExecutor module - Replace direct OpenAI client calls with unified executor pattern - Introduce LlmExecutor to coordinate throttle, retry, and fallback mechanisms in a single place - Update LlmClient to delegate execution to LlmExecutor - Add debug logging for LLM completion operations - Update test assertions to use new executor methods
- Introduce LlmPoolConfig for managing summary, retrieval, and pilot client configurations - Add comprehensive retry, throttle, and fallback mechanisms - Implement unified metrics configuration with LlmMetricsConfig, PilotMetricsConfig, and RetrievalMetricsConfig - Add cost calculation capabilities for LLM operations - Consolidate existing LLM-related config types into new modules - Update main Config struct to include llm and metrics fields feat(metrics): implement central metrics hub for unified collection - Create MetricsHub for collecting LLM, Pilot, and Retrieval metrics - Add thread-safe atomic metric tracking for multi-threaded usage - Implement comprehensive reporting capabilities - Add support for token tracking, latency measurement, and cost estimation - Include metrics for rate limits, timeouts, and fallbacks
Add query_hash and path_hash methods to PilotContext for generating hashes used in feedback learning system. Introduce feedback learning capabilities to LlmPilot including: - Add optional PilotLearner field to LlmPilot struct - Implement with_learner and with_feedback_store builder methods - Add helper methods has_learner, learner, and record_feedback - Integrate learner adjustment logic in decision making process - Apply confidence adjustments based on historical feedback data - Skip interventions when learner suggests low historical accuracy The feedback system allows the pilot to learn from past decisions and improve future recommendations through confidence adjustments and strategic intervention skipping.
- Add comprehensive design document for Feedback Learning system including architecture, core components, and integration details - Implement detailed documentation covering FeedbackRecord, FeedbackStore, and PilotLearner components - Add usage examples demonstrating feedback collection and learning - Include system diagrams showing data flow and component interactions - Document configuration options and learning strategies - Provide example code showcasing feedback learning implementation
…arning example - Remove unused DocumentTree import from vectorless::document module - Reorganize imports for better grouping and readability - Move PilotConfig import to appropriate location with other pilot-related imports - Rename persistent_store variable to _persistent_store to indicate it's intentionally unused
Remove the star call-to-action message and update the early development warning by removing the ⭐ emoji prefix. docs(architecture): enhance diagram with comprehensive system design Update architecture.svg to include: - Expand canvas size from 720px to 800px height - Add detailed configuration section with TOML settings - Include LLM executor with throttle control and retry mechanisms - Add query decomposition and pilot guidance components - Implement unified metrics hub with LLM, pilot, retrieval, and feedback statistics - Add feedback learning loop from user feedback to decision adjustment - Include design philosophy section highlighting zero vectors, algorithm+LLM hybrid approach, feedback learning, and multi-turn support - Replace keyword/semantic/LLM strategies with keyword-only, BM25, and hybrid scoring strategies - Update workspace to include feedback store component
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.