How to query index knowledge base and full knowledge base

[tommywolfheart]

Hi.

I am implementing a RAG system. May I please find out how I go about implementing a system that can query both the knowledge base I created as well as the knowledge base that comes with the LLM. I have the following code:

# Parse and chunk up data
index = VectorStoreIndex.from_documents(documents)

# Create query engine from index
query_engine = index.as_query_engine()

response = query_engine.query("What is the latest customer's reference number?")
response = query_engine.query("Who is Paul Graham?")

This is quite good at responding when the question is related to the documents I used to generate the index. However when I ask something outside of the document, it gives a terrible answer. So what is the correct approach/architecture to allow the system to give good responses in either case?

Thank you.

[tommywolfheart]

Decided to go with agentic RAG.