When sufficient context is not available in the RAG Knowledge base (stored in the Vector DB) for an incoming question, we would need to detect this and return a suitable answer to the question being asked. For e.g. answering: "Sufficient context is not available to answer this question" would be better than providing a wrong answer from non-relevant "context".
This detection will also need to be user-configurable - perhaps a confidence estimate on the "similarity" of the top-k relevant vectors returned for the end-user's question? Investigate the availability of such a metric and include it in the RAG serve app.