Skip to content

citations, contextual retrieval updates, ui, adaptive querying, history#52

Open
manasmhasakar wants to merge 2 commits intogeorgia-tech-db:mainfrom
manasmhasakar:manas/extra-credit-updates
Open

citations, contextual retrieval updates, ui, adaptive querying, history#52
manasmhasakar wants to merge 2 commits intogeorgia-tech-db:mainfrom
manasmhasakar:manas/extra-credit-updates

Conversation

@manasmhasakar
Copy link

This PR adds six major enhancements to TokenSmith: detailed citations, contextual chunk retrieval, adaptive query planning, conversation memory, performance optimizations, and a Streamlit web interface. These improvements significantly enhance answer quality, user experience, and system performance.

Core Enhancements

  • Citation System: Page numbers, chapters, and sections displayed with each answer
  • Contextual Retrieval: New ContextualRetriever with neighbor expansion and cross-reference boosting
  • Adaptive Query Planning: HeuristicQueryPlanner adjusts retrieval based on question type
  • Conversation Memory: Maintains 5-turn history for contextual follow-ups
  • Performance: Parallel embedding + incremental indexing (reduces rebuild time from ~3.5 hours)
  • Web UI: Streamlit interface with multiple chat sessions, history, and citation display

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant