2026
Support-RAG
A retrieval system you can point at your own documents: upload markdown, HTML, tickets, a changelog, or an OpenAPI spec, then ask questions and get answers that cite the passages they came from.
cat problem.md
The problem
Most retrieval demos answer plausibly and cite nothing, which makes them impossible to trust and impossible to debug. When an answer is wrong you cannot tell whether retrieval missed the passage or the model ignored it.
Pure vector search also has a well-known blind spot. It handles paraphrase well and exact identifiers badly, so a question about a specific error code or endpoint name often retrieves everything except the page that names it.
The third constraint was cost. The whole system had to fit on free tiers, including a 512 MB container, without quietly degrading answer quality.
tree --architecture
Architecture
Your files Question
md / html / pdf |
tickets / changelog v
openapi +-----------+
| | Vector | Qdrant
v | search |
structure-aware +-----+-----+
chunking |
(headings + path) | RRF fusion
| +-----+-----+
v | BM25 |
embeddings ---> Qdrant | keyword |
+-----+-----+
|
optional cross-encoder
rerank
|
v
LLM router: Groq -> Gemini
retries, circuit breaker
|
v
answer + citation audit
SSE token streamHybrid retrieval fuses vector and keyword results, then the answer is audited against the context it was given.
| layer | technology | responsibility |
|---|---|---|
| Retrieval | Qdrant + BM25, fused with RRF | Semantic recall plus exact-term precision in one ranked list |
| Ingestion | Structure-aware chunker, connectors | Splits on headings and prepends title and heading path to each chunk |
| Generation | Groq primary, Gemini fallback | LLM router with retries and a circuit breaker |
| Caching | Redis, semantic cache | Near-duplicate questions served from cache, invalidated on re-ingest |
| API | FastAPI, SSE | Streaming tokens followed by a meta event carrying citations |
| Quality | Hit@K, MRR, nDCG, Recall, faithfulness | Retrieval and answer evaluation against a golden QA set |
git log --decisions
Key decisions
Hybrid retrieval instead of vectors alone
Vector search and BM25 run in parallel and their rankings are fused with reciprocal rank fusion. Semantic search finds the paraphrase, keyword search finds the exact identifier, and fusion means neither one has to be right on its own.
Chunks carry their heading path
Splitting on headings and prepending the document title and heading path to each chunk gives a retrieved passage enough context to stand alone. A chunk that reads 'set this to true' is useless; one that reads 'Configuration > Reranking > set this to true' is not.
Every answer is audited against its context
A citation audit flags any marker the model produced that was not actually in the retrieved context. It turns a silent hallucination into a visible, loggable event.
The reranker is off in Docker on purpose
A cross-encoder reranker measurably improves ordering and does not fit in 512 MB alongside everything else. It is gated behind a flag and local extras, so the hosted build stays inside the free tier and a local run can turn it on.
Uploads are isolated per visitor
Each browser session gets its own upload folder, and retrieval sees the shared corpus plus that visitor's files only. Two people can upload a file with the same name on a public demo without colliding.
The semantic cache is invalidated by ingest
Caching near-duplicate questions is a large latency win and a correctness hazard. Tying invalidation to corpus changes means re-ingesting new documents cannot leave a stale answer in front of the next visitor.
cat outcome.md
Outcome
- Live on Render at support-rag-p177.onrender.com, containerized with Docker and validated by 35 unit and integration tests.
- Retrieval quality is measured, not asserted: Hit@K, MRR, nDCG, and Recall for retrieval, plus answer faithfulness against a golden QA set.
- Runs entirely on free tiers, including a 512 MB container, with optional Langfuse tracing and structured per-stage timings.