Description
Add a second analyze endpoint that accepts question, generated answer, and pre-retrieved chunks directly — bypassing RAGBench and ChromaDB entirely. This is the integration point for anyone with an existing RAG system (OpenSearch, Pinecone, Weaviate, etc.) who wants to run the forensics layer on their own pipeline's output.
The forensics services themselves don't change — this is purely a new router entry point.
Implementation details
class CustomChunk(BaseModel):
chunk_id: str
text: str
score: float
class CustomAnalyzeRequest(BaseModel):
question: str
answer: str
chunks: list[CustomChunk]
- Add POST
/analyze/custom in routers/analyze.py
- Convert
CustomChunk → RetrievedChunk, pass directly to forensics services
- Return same
AnalyzeResponse shape as /analyze
- Add
README_INTEGRATION.md with minimal Python example:
import requests
chunks = [
{"chunk_id": "doc_42_chunk_3", "text": "...", "score": 0.87},
]
response = requests.post("https://your-deployment.railway.app/analyze/custom", json={
"question": "What is the refund policy?",
"answer": "Refunds processed in 5-7 days.",
"chunks": chunks
})
print(response.json())
Acceptance criteria
TDD
tests/test_analyze_custom.py (assert ChromaDB retriever mock NOT called):
- Valid request returns 200 with
AnalyzeResponse shape
- Empty
chunks returns 422
- Missing
question returns 422
- Chunk missing
score returns 400
- All six forensics dimensions present in response
attribution_map length matches sentence count in provided answer
- ChromaDB retriever is NOT called
Description
Add a second analyze endpoint that accepts question, generated answer, and pre-retrieved chunks directly — bypassing RAGBench and ChromaDB entirely. This is the integration point for anyone with an existing RAG system (OpenSearch, Pinecone, Weaviate, etc.) who wants to run the forensics layer on their own pipeline's output.
The forensics services themselves don't change — this is purely a new router entry point.
Implementation details
models.py:/analyze/custominrouters/analyze.pyCustomChunk→RetrievedChunk, pass directly to forensics servicesAnalyzeResponseshape as/analyzeREADME_INTEGRATION.mdwith minimal Python example:Acceptance criteria
/analyze/customwith valid inputs returns 200 withAnalyzeResponsechunkslist returns 422questionoranswerreturns 422scorefield returns 400/analyzeREADME_INTEGRATION.mdexists with working minimal Python exampleTDD
tests/test_analyze_custom.py(assert ChromaDB retriever mock NOT called):AnalyzeResponseshapechunksreturns 422questionreturns 422scorereturns 400attribution_maplength matches sentence count in provided answer